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This  Fined  Report  is  divided  into  four  sain  sections.  In  the  first 
section,  I  describe  in  some  detail  a  subset  of  the  enpiriced  research  I 
have  done  vdth  sy  collaborators  \sider  the  contract.  Rather  than  attempt 
to  describe  every  study,  I  have  described  selected  studies  in  detail. 
These  studies  shew  some  of  the  range  of  work  we  have  aooenplished  during 


the  contract.  In  the  second  section,  I  present  a  list  of  the  publications 
that  have  ensued  frem  1985  to  1988,  as  well  as  publications  that  are  in 
press.  These  are  the  years  during  vhich  the  contract  was  held.  In  the 
third  section,  I  list  presentations  I  have  given  during  the  years  of  the 
contract.  And  in  the  fourth  and  final  section,  I  give  the  final  budgetary 


accounting  far  the  contract,  j' 
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Section  1 

Coping  with  Novelty  in  Human  Intelligence: 

An  Empirical  Investigation^ 

Kuhn  (1970)  has  observed  that  the  tsnily  influential  scientists  are 
those  who  see  things  differently  frcn  the  way  others  see  things:  They  are 
the  scientists  who  create  new  paradigms  rather  than  following  existing 
ones.  In  other  words,  they  can  take  an  old  problem  and  see  it  in  a  way 
that  is  different  fron  the  way  other  scientists  see  it.  For  exanple, 
Chcnsky  (1965)  created  a  new  paradigm  in  linguistics  vhen  he  introduced 
the  concept  of  transformationed  graranar,  seeing  language,  a  familiar 
entity,  in  a  way  no  one  had  seen  it  before. 

Kuhn's  observation  applies  not  only  to  science.  In  the  investment 
world,  those  investors  vho  are  oonsld^ed  to  be  xaiusually  successful  by 
virtue  of  the  outstanding  performance  of  their  investment  portfolio  are 
almost  edways,  to  greater  or  lesser  degree,  "contrarian"  investors 
(Dreanan,  1982) .  The  reason  for  this  phenomenon  is  that  a  good  investment 
generedly  stops  being  good  just  as  soon  as  everyone  recognizes  that  it  is 
good.  In  the  stock  market,  for  exasple,  the  only  way  to  make  money  on  a 
stock  is  to  recognize  that  it  is  undervalued  before  everyone  else 
recognizes  it  to  be  isidervalued.  As  soon  as  everyone  else  recognizes  that 
fact,  they  buy  the  stock  and  it  is  no  longer  undervalued,  as  its  price  has 
now  risen. 

The  observations  of  scholars  about  science  and  the  investment  world 
apply  in  everyday  life  as  well.  When  questioning  someone  in  order  to 
establish  her  intelligence  or  general  conpetenoe,  we  often  throw  the 
person  a  "curve  bedl,"  meaning  that  we  ai^  her  a  question  that  is 
different  in  kind  traa.  the  question  that  is  typically  asked  of  people  in  a 
given  situation.  If  she  is  able  to  answer  ordinary  questions,  but  not 
ones  that  require  soma  degree  of  flexible  thinking  or  insight,  we  may 
conclude  that  she  is  only  someidiat  intelllgenoe  or  competent,  or  even  that 
she  is  pedestrian.  Indeed,  in  their  studies  of  people's  oono^Ttions  of 
intelligence,  Sternberg,  Ocnway,  Ketron,  and  Bernstein  (1981)  found  that 
people  valued  as  intelligent  the  ability  to  see  problems  in  a  new  and 
different  way. 

Psychologists  as  well  as  laypersons  have  recognized  for  seme  t-ime  the 
importance  of  coping  with  novel  kinds  of  tasks  or  situations  to 
intelligence  for  a  long  time.  For  exanple,  ptychcmetric  tests  of 
intelligenoe  often  iixd.\x3e  items  that  measure  a  person's  ability  to  cope 
with  novelty,  and  the  theory  of  intelligence  proposed  by  Cattell  and  Horn 
(Cattell,  1971;  Horn  and  Cattell,  1966)  distinguishes  between  two  kinds  of 
intelligence  -crystallized  and  fluid.  Crystedlized  intelligenoe  is  the 
set  of  products  representing  accumulations  of  the  processes  of  fluid 
intelligenoe,  \^ch  are  the  processes  used  in  thinking  in  new  and 
different  ways.  Tests  measuring  fluid  ability  would  be  abstract 
analogies,  classifications,  series,  or  matrices,  all  of  vftilch  require  one 
to  cope  with  novel  kinds  of  problems.  Baaheim  (1974)  viewed  the  ability 
to  cope  with  relative  novelty  as  the  key  ingredient  of  intelligenoe,  a 
view  taken  as  well  by  Sternberg  (1981,  1982,  1985) ,  whose  trlarchic  theory 
of  intelligence  holds  that  the  ability  to  cope  with  relative  novelty  is 
one  of  three  key  ingredients  to  human  intelligenoe  (see  Sternberg,  1985) . 
Piaget  (1972) ,  too  in  a  different  tradition,  measured  intelligenoe  by 
presenting  children  with  novel  kinds  of  tasks,  such  as  the  balanoe-beam 
task  or  any  of  a  number  of  conservation  tasks. 
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Althcu^  laypersons  and  psyctelogists  edUce  hacve  ireoognized  the 
inpcartance  of  coping  with  norvelty  to  intelligence,  oddly  enou^,  there 
e^jpears  to  be  no  direct  experimental  test  of  the  hypothesis  that  a 
relatively  novel  version  of  a  given  task  will  provi^  significantly  better 
assessment  of  a  person's  intelligence  than  will  a  non-novel  version  of 
that  task.  Althcu^  a  nunber  of  investigators,  including  ourselves 
(Sterrbexg  &  Gastel,  in  press) ,  hacve  shown  the  relevance  of  the  ability  to 
cope  with  novelty  to  inti^ectual  performance  (see  adso  Davidson  & 
Sternberg,  1984;  Sternberg,  1982;  Sternberg  &  Davidson,  1982;  Tetewsky  & 
Sternberg,  1986) ,  the  line  of  argument  has  been  that  tasks  that  require 
coping  with  novelty  correlate  fairly  hi^y  with  peychcnetric  tests  of 
fluid  intelligence.  None  of  these  studies  have  shown  significantly  hicber 
correlations  for  the  tasks  requiring  coping  with  novelty  than  for  tasks 
that  do  not  require  this  ability. 

Ihe  purpose  of  this  investigation  was  directly  to  test  the  hypothesis 
that  the  ability  to  cope  with  a  relatively  novel  version  of  a  task  is  more 
hi^ily  related  to  fluid  intelligence  than  is  the  ability  to  cope  with  a 
non-novel  version  of  that  task.  In  other  words,  this  study  seeks  directly 
to  test  the  hypothesis  that  the  ability  to  cope  with  novelty  is  a  key 
ingredient  of  intelligenoe.  We  tested  this  hypothesis  ty  presenting 
subjects  with  a  verification  task  that  reqitires  subjects  to  make  either  a 
familiar  presqpposition  or  a  novel  oaie.  We  hypothesized  that  requiring 
the  subject  to  reason  with  a  novel  presupposition  would  increase  the 
extent  to  which  a  task  measures  flxdd  intelligenoe. 

Subjects 

Subjects  in  the  main  stuty  were  50  Yale  undergraduates  (17  females,  33 
males) ,  all  of  vhaa  received  course  credit  for  their  participation  in  the 
experiment,  which  took  roughly  two  hours.  In  addition,  60  summer  studmits 
at  Yede  provided  ratings  in  a  prestuty  that  were  used  in  order  to  validate 
the  materials  for  the  main  study. 

Materials 

There  were  two  basic  kinds  of  materials;  a  verification  task  and 
psychometric  ability  tests.  The  verification  tasks  ccnprised  the  main 
experimented  stixuli,  vhereas  the  ability  tests  were  used  to  validate  the 
main  hypothesis  of  the  stuty  regarding  the  relationship  of  ocping  with 
novelty  with  fluid  intelligenoe. 

The  verlficatinn  taste,  ihe  verlficaticn  task  consisted  of  232 
statements,  each  of  vhich  a  subject  had  to  identify,  as  rapidly  as 
possible,  as  being  either  true  or  fedse.  Statements  were  grouped  into 
sets  of  from  five  to  ei^t  individual  statements,  and  each  of  these  sets 
was  preceded  by  a  presupposition,  which  subjects  were  instructed  to  assume 
was  true.  For  half  of  34  different  sets  of  items,  the  presupposition  was 
one  that  was  true  in  the  everyday  world;  for  the  other  hedf  of  the  item 
sets,  the  presupposition  was  oounterf actual.  A  given  set  of  items  was 
pair^  with  both  a  familiar  and  a  oounterfactued  presupposition.  Half  of 
the  subjects  saw  a  given  set  of  items  with  the  familiar  presupposition, 
and  half  saw  that  set  with  the  oounterfactual  presupposition.  Table  1 
shews  seme  representative  sets  of  items  frem  the  experiment,  paired  with 
both  their  familiar  and  their  oounterfactuud  presupposition.  Of  course,  a 
given  subject  saw  each  of  the  items  with  only  one  of  the  two  possible 
presuppositions  and  did  not  see  the  keyed  answer. 

Table  1  illuistrates  that  for  statements  with  familiar  presuppositions, 
the  correct  answer  to  the  verification  task  is  the  same  as  it  would  be  in 
the  ordinsuy,  everyday  world.  Hewever,  for  item  sets  with  a 
oounterfactual  presupposition,  the  key^  answer  was  the  same  as  that  in 
the  everyday  world  for  only  half  the  total  number  of  items.  For  this  half 
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o£  the  Itans  with  the  csunterfactual  pr^ipposition,  the  presi^^poeltian 
was  Irrelevant  to  verification  of  the  statement.  For  the  other  half  of 
the  items  with  a  counterfactual  presipposition,  however,  the 
presupposition  was  relevant,  and  hence  changed  the  keyed  response.  Note, 
then,  that  the  task  version  with  the  ccunterfactual  presupposition 
required  a  oert2dn  kind  of  flexibility  that  the  task  version  with  the 
familiar  presupposition  did  not  require,  as  the  former  version  of  the  task 
required  a  different  answer  from  the  normal,  everyday  one  for  half  the 
items,  whereas  the  version  of  the  task  with  familiar  presi;ppo6itians  never 
requirai  an  answer  that  was  different  from  the  normal,  everyday  one. 


Insert  Table  1  about  here 


Ihe  subjects  in  the  prestuc^  were  used  to  verify  that  our  keying  of 
responses  was  the  same  as  that  which  would  be  rendered  by  the  subjects 
actually  performing  the  task.  The  prestufy  subjects  were  asked  to  assume 
that  each  presipposltion  was  true,  and  then  to  mark  each  following 
statement  as  true  or  false,  based  on  the  presipposition.  Subjects  were 
also  asked  to  provide  a  confidence  rating  for  each  statement  indicating 
how  sure  they  felt  they  were  of  the  correctness  of  their  answer  (on  a 
l»law,  7»high  scale) .  Questicnnzdres  ocnslsted  of  36  sets  of  items,  half 
with  famili^  presippositions  and  hedf  with  oounterfactued 
prescppositions.  TWo  different  forms  of  the  questionnedre  were 
administered  to  two  grtxps  of  subjects.  The  questionnaires  differed  in 
that  items  having  ccunterfactual  presi:ppositlons  in  the  first  form 
2^]peared  with  factual  presuppositions  in  the  second  form,  and  vice  versa. 
Although  the  presuppositions  differed  between  forms,  the  statements 
following  them  were  identical.  Items  were  retained  for  the  final  stucty 
only  if  there  was  clear  agreement  regarding  what  should  be  the  keyed 
answer. 

Whereas  in  the  prestudy  the  items  were  administered  with  no  time 
pressure,  subjects  in  the  main  stuc^  were  aware  that  they  would  be  timed 
in  performing  the  tasks,  and  were  told  to  be  as  accurate  as  possible 
without  making  errors. 

Psychometric  tests.  All  subjects  in  the  finri  study  received  four 
psychometric  ability  tests  in  a  fixed  order:  (a)  the  Letter  Sets  Test 
from  the  French  Kit  of  Reference  Tests  for  Cognitive  Factors  (Ekstrcm, 
French,  &  Harman,  1976) ,  (b)  the  Syllogisms  Test  from  the  Califomia  Test 
of  Menbd  Maturity,  Level  5  (Sullivan,  dark,  &  Tiegs,  1963) ,  (c)  the 
Cattell  Culture-Erir  Test  of  g.  Scale  3,  Form  A  (Cattell  &  Cattell,  1963) , 
and  (d)  the  Finding  A's  Test  from  the  French  Kit  of  Reference  Tests  for 
Cognitive  Factors  (Ekstrcm,  French,  &  Harman,  1976) .  The  first  three  of 
these  ability  tests  are  well-validated  measures  of  flviid  intelligence 
incluided  for  purposes  of  convergent  validation  of  the  experiments 
hypothesis.  Ihe  last  test  was  a  test  of  peroeptual-motor  speed,  incluided 
for  purposes  of  discriminant  validation. 

Pigla? 

The  medn  dependent  variable  was  re^xmise  time  to  each  of  the 
statements  to  be  verified.  A  secondary  dependent  variable  was  error 
rate.  Psychcmetric  ability-test  scores  were  used  as  dependent  variables 
for  purposes  of  convergent  and  discriminant  validation.  The  aain 
lnd^)endent  variables  were  presupposition  type  (novel,  non-novel)  and  type 
of  response  (true,  false) . 

Procedure 

In  the  prestudy,  subjects  received  sets  of  statements  pedred  with 
either  familiar  or  ccunterfacturi  presippositions,  and  had  to  indicate  for 
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each  statesoent  vhether  It  vas  tirue  or  false,  given  the  prest^^ipositlan,  anr^ 
also  vhat  their  oonfidenoe  rating  was  (on  a  1-7  scale)  for  their  labeling 
of  the  statement  eis  true  or  fzQse.  Item  sets  were  presented  in  random 
order.  In  the  main  ea^ieriinent,  subjects  also  received  items  sets  in 
random  order,  foUcMed  at  the  end  of  the  esqperijnent  by  the  psychometric 
ability  tests  in  fixed  order  (in  a  separate  group  session) .  At  the 
beginning  of  the  experimental  session,  each  subject  was  orally  briefed  on 
the  general  content  of  the  eaperiment,  and  then  presented  with  a  consent 
form  to  sign.  Stimuli  were  presented  on  a  Macintosh  parannai  ocnputer. 
First,  instructions  appeared  on  the  ocnputer  screen.  Then  the  selects 
saw  four  practice  items.  On  these  prac^oe  itans,  the  ocnputer  informed 
the  subject  whether  each  response  was  correct  or  incorrect.  However,  no 
feedback  was  provided  for  the  rest  of  the  items.  Half  way  threu^  the 
cenputer  session,  subjects  were  given  an  opportuni'ty  for  a  break,  after 
which  they  continued  with  the  second  part  of  the  experimental  items. 
Ability  tests,  vhioh  were  all  in  papei>and-pencil  format,  were 
administered  aftsr  all  verification  statements  were  oonpleted. 

Before  each  subject  began,  all  item  sets  viere  randomized  and  all 
statements  within  each  item  set  were  randomized  in  order.  Each  subject 
saw  all  34  item  sets,  including  232  statements.  Odd-numbered  subjects  saw 
items  from  a  first  batch  with  ocunterfactual  presuppositions  and  items 
from  a  second  batch  with  familiar  presuppositions;  evenmumbered  subjects 
saw  the  reverse  pairing. 

Each  item  appeared  as  follows:  First  the  presupposition  was 
presented,  prefaced  by  the  phrase  "Assume  that. ..."  The  subject  had  as 
long  as  he  or  ^  desired  to  read  the  presupposition.  When  rear^,  the 
subject  would  press  the  space  bar,  vhich  would  initiate  the  presentation 
of  a  statement  to  be  verified.  The  subject  responded  to  the  statement  by 
pressing  a  k^  on  the  ocnputer  board.  !Ihe  suli^ect's  response  and  re^xnse 
time  were  recorded.  After  a  half-second  delay,  the  next  statement  would 
eppear,  and  so  on.  All  statements  appeared  iix  bold  face  type,  with  the 
presupposition  written  above  in  normal  type  in  parentheses.  The 
presupposition  appeared  for  each  statement  so  the  subjects  would  not  have 
to  menendae  the  presuppositions. 

Results 

Table  2  presents  the  means  far  each  of  the  madn  conditions,  and  Table 
3  presents  the  results  of  the  anadyses  of  variance.  As  would  be  expected, 
novel  items  were  more  difficult  than  familiar  items,  as  indexed  both  by 
hitler  response  times  and  hi^ier  error  rates.  False  items  were  more 
difficult  than  true  ones.  The  corzelation  of  response  times  with  error 
rates  across  items  was  .26  (p  <  .001). 


Insert  Tables  2,  3,  and  4  about  here 


Table  4  shews  correlations  of  re^mse  times  and  error  rates  with 
psychcmetric  ability  test  scores.  The  task  shows  good  convergent  and 
discriminant  vadidity,  su^pesting  that  it  is  a  good  measure  of  fluid 
intelligence  but  that  it  does  not  measure  peroeptuad-motor  speed. 

The  usefulness  of  the  verificaticn  task  for  measuring  fluid  abilities 
is  shown  by  a  stepwise  sultiple  regression  of  the  psychcmetric  ability 
tests  on  the  experimental  task  scores.  For  both  novel  and  non-novel 
items,  two  of  the  three  ability  tests  enter  into  the  nultlple  regression: 
letter  Sets  and  Syllogisms.  For  novel  re^)onse  times,  the  multiple 
correlation  was  .73  with  the  two  ability  tests,  p  <  .001.  For  the 
non-novel  items,  the  multiple  correlation  was  .68,  p  <  .001.  Thus,  the 
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verification  task  e$:pears  to  ta^  viell  the  sane  oonstrvact  as  is  tapped  by 
conventional  fluid-ability  tests. 

Ihe  oritixal  correlations  for  the  experiment  are  those  that  test  the 
hypothesis  that  the  novel  condition  provided  better  assessment  of  fluid 
intelligence  but  not  peroeptual-inotar  speed  than  did  the  non-^iovel 
condition.  In  order  to  test  this  hypothesis,  a  difference  score  was 
conputed  for  each  subject  «her^  the  subject's  average  response  tine  for 
non-novel  was  subtracted  from  his  or  her  average  response  tine  for  novel 
items.  Ihis  difference  score  reflects  the  incrapwnt  in  time  to  respond 
for  novel  Items  (i.e. ,  items  with  ocunterfactual  preseppositions)  over 
non-novel  itans  (i.e.,  items  with  familiar  presippositicais) .  The 
correlation  of  this  difference  score  with  the  Letter  Sets  Test  was  -.34,  p 
<  .02.  The  correlation  of  the  difference  score  with  the  Syllogisms  Test 
was  -.38,  p  <  .01.  The  correlation  with  the  cattell  Test  was  -.15,  p  > 
.05.  Fiially,  the  correlation  with  the  Crossing  out  A's  Test  was  .04,  p  > 
.05.  Thus,  all  three  correlations  of  the  difference  score  with  the 
psychometric  ability  tests  were  negative,  as  would  be  predicted  by  the 
main  hypothesis  of  the  eaperiment.  TWo  out  of  the  three  correlations  were 
statis^cally  significant.  The  correlation  of  the  peroeptaMl-motor  speed 
test  with  the  difference  score  was  trivially  positive.  In  other  words, 
the  nadn  hypothesis  of  the  study  was  generally  confirmed. 

Discu^ion 

The  goal  of  this  eaperiment  was  directly  to  test  the  hypothesis  that 
coping  with  novelty  is  an  important  aspect  of  intelligence.  The  test  of 
the  hypothesis  was  direct  in  the  sense  that  subjects  received  both  a 
relatively  novel  and  a  relatively  non-novel  version  of  a  sentence 
verification  task,  and  it  was  shewn  that  the  difference  score  oenparing 
response  times  for  the  two  tasks  was  significantly  correlated  with 
psychametrlc  measures  of  fluid  abilities.  In  other  words,  the  increment 
in  tine  associated  with  the  novel  versus  non-novel  version  of  the  task 
significantly  predicted  fluid  ability.  We  believe  that  this  result 
constitutes  the  most  direct  test  to  date  of  the  hypothesis  that  coping 
with  novelty  is  an  important  aspect  of  fluid  inteiligence,  edthough  it 
certainly  does  not  constitute  the  only  test.  Moreover,  we  recognize  that 
no  single  task  can  provide  a  broad  test  of  the  hypothesis,  as  there  are 
nary  kinds  of  novelty,  and  the  kind  of  novelty  in^lved  in  the 
verification  task  with  counterfactual  presippositions  is  only  one  of  many 
possible  kinds. 

Our  intuition  that  the  ability  to  see  things  in  a  new  and  different 
way  is  an  important  ingredient  of  intelligence  is  oonfimed  by  this 
study.  This  is  not  to  say,  however,  that  edl  intelligent  people  have  the 
ability  to  cope  well  with  novelty.  To  the  contrary,  some  people  nay  be 
very  intelligent  just  so  long  as  their  everyday  presippositions  can  be 
taken  for  granted.  The  intelligence  with  which  they  perform,  however,  may 
breakdown  when  novelty  is  introduced.  Oonversely,  scmie  people  may  be  at 
their  best  in  coping  with  novelty,  and  look  less  intelligent  in  comparison 
with  others  vhen  confronted  with  more  mundane  tasks  (Sternberg,  1985) .  In 
understanding  intelligence,  then,  we  need  to  view  it  in  its  diversity,  and 
to  recognize  that  intelligence  has  many  aspects,  an  important  one  of  vhich 
is  the  ability  to  cope  with  relative  novelty. 
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If  Dancers  Ate  Iheir  Slioes: 

Inductive  Reaiscning  with  Factuzd  and  Ocunterf actual  Premises^ 

In  our  everyday  lives,  we  rust  frequently  make  inductive  judgments. 
Most  of  these  judgments  are  routine,  requiring  standard  reasoning  from 
familiar  presnises.  But  occasionally  we  rust  afifr.m>R  that  scmetdiing  is  true 
about  which  we  may  have  little  or  no  information,  or  which  may  even  be 
ocunterf  actual.  Suppose,  for  example,  ve  view  Nicaragua  as  having  a 
socialist  government  interested  only  in  self-determination.  Does  the 
standard  U.S.  government  analogy  between  Nicaragua  and  Cuba  still  hold? 

Or  sippose  we  believe  that  a  leading  clinical  psychologist  has  three 
maladjvisted  children.  Can  this  psychologist's  clinical  advica  still  be 
trusted? 

An  exarple  of  the  need  for  such  potentially  oounterfactuad.  inductive 
reasoning  occasionally  arises  in  detective  work,  as  shown  in  Isaac 
Asimov's  (1979)  short  story,  "The  Singing  Bell."  The  question  faced  by 
the  world-famous  professor  of  extraterrestrial  phencnena.  Dr.  lh±h,  is 
vhether  Louis  Peyton  was  on  the  moon  recently,  where  he  eOlegedly  murdered 
Albert  Oomwell.  Solving  the  problem  requires  Dr.  urth  to  reason  about 
the  effects  of  unfamiliar  gravitational  levels. 

The  motive  for  the  murder  would  have  been  to  obtadn  a  large  quantity 
of  a  rare  and  valuable  life  farm  called  the  Singing  Bell.  Peyton  denies 
both  the  murder  and  having  been  on  the  moon.  Dr.  urth  asks  the  accused, 
Peyton,  vhether  he  respects  Singing  Bells.  Peyton  replies  that  he 
does — too  much  to  break  one.  At  this  point,  Peyton  gently  strokes  the 
Singing  Bell  he  is  holding.  Dr.  Urth  suddenly  commands  Peyton  to  toss  the 
Singing  Bell  to  him.  Peyton  automatically  tosses  the  Bell.  It  travels  a 
short  arc  one-third  the  way  to  Urth,  curves  downward,  and  shatters  on  the 
floor.  Dr.  Urth  then  concludes: 

Surely  the  matter  is  now  obvious.  The  fact  that  Mr.  Peyton  could  so 
egregiously  misjudge  the  toss  of  an  ci^ect  he  obviously  valued  so 
hl^ily  could  mean  only  that  his  muscles  are  not  yet  readjvisted  to  the 
pull  of  Earthly  gravity.  It  is  my  professional  opinion,  Mr. 

Davenport,  that  your  prisoner  has,  in  the  last  few  days,  been  away 
from  Earth.  Hie  has  either  been  in  space  or  on  some  plaretary  object 
considerably  smaller  in  size  than  the  Earth — as,  for  example,  the 
Moon.  (p.  237) 

In  the  above  story.  Dr.  Urth  thinks  in  terms  of  what  would  follow  if 
an  imfamiliar  situation  were  true.  Such  reasoning  with  unfamiliar 
information  can  be  readily  incorporated  into  indu^ve  reasoning 
problems.  For  example,  assume  that  dinosaurs  are  kinds  of  fruit  juices. 
Given  this  premiise,  vhat  word  would  ccme  next  in  the  following  series: 
SUBSTANCE,  LIQUID,  EKENK,  ITOTT  JUICE,  (a)  BRCNTOSfiUTOS,  (b)  CDFEEE,  (c) 
VEGETABLE,  (d)  GLASS? 

CXxr  attaopt  to  eiqplore  reasoning  with  different  kinds  of  information 
is  part  of  an  ongoing  prxjgram  of  research  in  vhich  we  eoplore  the 
relations  between  task  novelty  and  information  processing  in  tasks 
requiring  human  intelligence.  This  larogram  of  research  is  motivated  by 
the  notion  that  the  ability  to  deal  with  relative  ta^  novelty  is  a 
particularly  crucial  aspect  of  human  intelligence  (Sternberg,  1931,  1985; 
see  also  Raaheim,  1974) .  In  other  words,  tasks  that  present  subjects  with 
intermediate  amcunts  of  novelty  may  be  gcxsd  measures  of  intelllgenoe. 

This  assertion  is  part  of  the  triarchic  theory  of  intelligence 
(Sternberg,  1985) .  According  to  this  theory,  int^ligence  ocmprises 
ocBponents  of  information  processing  that  people  employ  in  deciding  how  to 
adapt  to,  shape,  or  select  their  environment.  Particularly  relevant  to 
the  present  esperiment  is  the  experiential  subtheory,  which  asserts  that 
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intelllgenoe  is  xaeasured  to  the  eictent  that  a  particular  task  is 
relatively  novel  (as  in  the  present  esqieriinent)  or  requires  autonatization 
of  infomation  processing.  In  this  experiment  we  seek  to  test  the  novelty 
aspect  of  this  subtheory  through  the  use  of  oounterfactual  novelty  in 
inikictive  reasoning  probleoss. 

In  this  experiment  we  also  eoqilare  the  ability  to  sift  relevant  frcn 
Irrelevant  Infomaticn.  This  ability,  vAiich  we  can  selective  encoding, 
is  one  of  three  abilities  (the  others  are  selective  ocnbination  and 
selective  ccnparison)  iiyhich  we  have  proposed  as  basic  to  insic^ 

(Sternberg  &  Davidson,  1982,  1983).  This  theory  of  insic^  is  edso  part 
of  the  esqieriential  subtheory  of  the  triarchic  theory  of  intelligenoe. 

The  scheme  of  the  program  of  research  is  shown  in  Table  5.  Our 


Insert  Table  5  about  here 


research  on  induction  has  proceeded  throu^  four  overlaying  phases.  In 
the  first  phase,  we  and  others  attenpted  task  deocoposltion  to  urxierstand 
the  information-processing  conponents  underlying  performance  on  familiar, 
IQ-test-like  inductive-reasoning  items  (Sternberg,  1977;  steniberg  & 
Gardner,  1983 ;  see  also  MuIhoUand,  Pellegrino,  &  Glaser,  1980) . 
Performance  on  tasks  such  as  analogies,  claseiifications,  and  series 
oonpletions  was  deocnposed  into  its  elanentary  ocppcnents  of  processing, 
enabling  us  to  ascertain  the  latencies  of  such  conponents,  and  the 
strategies  into  which  they  were  ccnbined.  In  a  second  phase,  we 
investigated  tasks  that  were,  in  some  sense,  at  the  opposite  extreme, 
namely,  unfamiliar  and  un-IQ-test-like.  In  one  kind  of  problem,  the 
ooncepbial  projection  problem,  subjects  were  required  to  predict  the 
future  state  of  an  object,  given  inccnplete  information  about  its  present 
state.  Objects  in  these  esperiments  were  other^-worldly.  For  exasple,  the 
people  of  the  planet  tyron  oould  be  either  bom  young  and  die  young,  bom 
old  and  die  old,  bom  young  and  die  old,  or  bom  old  and  die  young 
(Steniberg,  1982;  see  eOso  Tetewsty  &  Sternberg,  1986) .  In  a  second  kind 
of  problem,  the  insicht  problem,  objects  hzxi  to  solve  what  appeared  to  be 
standard  mathematics  problems,  but  what  were  in  fact  insight  problems  that 
could  not  be  solved  by  routine  mathematiced  fomulae  (Davidson  & 

Steniberg,  1984;  Sternberg  &  Davidson,  1982).  In  a  third  phase,  we 
investigated  problems  that  were  familiar  but  un-IQ-test-like — indvx±ive 
predictions  into  the  future  (vhen  will  a  bottle  of  milk  spoil?)  and 
postdictions  into  the  past  (when  did  the  bottle  of  milk  ^»il?)  (Kalmar  & 
Sternberg,  1985) .  The  present  ejperiment  ryresents  work  in  a  fourth 
phase  of  research. 

In  the  fourth  phase,  we  have  been  investigating  prcblans  that  are 
IQ-test-li}QB,  but  unfamiliar.  Such  problems  have  the  theoreticad 
advantage  that  they  may  measure  the  ability  to  cope  with  relative  novelty, 
diich  seems  quite  imp(»±ant  to  intelligenoe  (Sternberg,  1985) ,  and  may 
adso  be  practical  for  use  both  in  laboratory  expuiments  and  on  tests  of 
human  intelligmioe,  broadly  defined.  These  problems  involve  a  mix  of 
items,  some  requiring  reasoning  based  on  facts  and  others  requiring 
reasoning  based  on  oounterfactual  ("novel")  premises.  In  a  first 
experiment  in  this  series,  Marr  and  Steniberg  (1986)  found  that  both 
gifted  and  nongifted  students  in  grades  6,  7,  and  8  gave  significantly 
iDore  attention  to  novel  information  than  to  familiar  infomation  in  the 
test  problems,  vAiich  were  analogies  preceded  by  a  cue  that  could  be  either 
novel  (e.g. ,  sparrows  play  hopsootch)  or  familiar  (e.g. ,  pistols  are 
weapons) ,  and  either  relevant  or  irrelevant  to  analogy  solution.  Gifted 
students,  however,  gave  significantly  less  attention  to  irrelevant  novel 
informaticn  than  did  nongifted  students,  but  did  not  differ  from  the 
nongifted  students  in  their  attention  to  relevant  novel  information. 
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The  Marr  and  Sternberg  (1986)  ea^)eriment  addressed  differences  in 
cutocnes  of  infomatlon  processing  of  factual  and  counterfactual  analogies 
for  gifted  and  nonglfted  children,  but  did  not  examine  the  question  of 
just  vhat  informatlcn  processing  Is  used,  nor  did  It  investigate 
infomatlrr.  processing  in  other  ^pes  of  problems.  Moreover,  the  work  was 
limited  t-o  grade-’school  children.  In  the  present  experiment,  we  seek  to 
oonfrnnc  head-on  the  question  of  hew  adults  process  Information  in  solving 
factual  and  counterfactual  analogies,  classifications,  and  series 
problems.  These  three  kinds  of  test  items  were  not  chos^  arbitrarily: 
They  are  the  three  inductive  Item  types  most  often  used  to  measure  general 
intelligence,  and  particularly,  so-called  fluid  intellectual  abilities 
(cf.  cattail,  1971;  Horn  &  Cattail,  1966).  By  investigating  three  related 
but  distinct  item  ^pes,  we  can  investigate  whether  the  results  generalize 
or  vhether  they  are  particular  to  each  Item  ^pe. 

Method 

Subjects 

Subjects  were  60  Yede  undergraduates — 17  males  and  43  females — In  an 
introductory  psychology  course.  Subjects  participated  teward  fulfillment 
of  a  course  requirement. 

Materials  were  of  two  basic  types,  experimental  tasks  and  psychcroetric 
tests. 

ExperliMTitai  taaiea.  Three  different  experimental  tasks  were  used: 
analogies,  classifications,  and  series  ooepletlons.  In  each  task,  half  of 
the  Items  were  uncued  and  h2df  were  precued.  Subjects  saw  each  Item  In 
either  precued  or  uncued  form,  but  not  both,  with  hedf  the  subjects  seeing 
a  given  Item  in  precued  form  and  half  in  urKued  form.  Each  task  Item  was 
afwociatad  with  a  precue  of  one  of  four  precue  types— familiar  relevant, 
familiar  irrelevant,  novel  relevant,  or  novel  irrelevant— which  appeared 
only  in  the  precued  condition.  Items  were  not  the  same  across  precue 
conditions.  Examples  of  each  of  these  ^pes  of  Items  for  each  task  are 
shown  in  Table  6. 


Insert  Table  6  about  here 


Each  subject  received  a  total  of  216  inductive-reasoning  items.  Those 
Items  that  appeared  in  uncued  form  were  simply  preceded  by  a  blank  field. 
Subjects  haKi  as  long  as  they  wanted  to  look  at  the  blank  field,  and  then 
pressed  a  button  vAilch  caused  the  induction  Item  to  appear.  Those  Items 
that  appeared  in  precued  form  were  preceded  by  a  premia  that  cculd  be 
either  familiar  or  novel,  and  either  relevant  or  irreletvant.  Subjects 
were  first  presented  with  the  precue,  and  were  given  as  long  as  they 
wished  to  read  It.  They  then  pressed  a  button,  which  resulted  in  the 
disappearance  of  the  precue  and  the  immediate  appearance  of  the  Induction 
Item. 

Items  were  equally  divided  among  verbal  anedogles,  classifications, 
and  series  ocnpletlons,  equally  divided  agzdn  between  uncued  and  precued, 
and  equally  divided  ag^  (for  the  precued  Itans)  among  familiar  relevant, 
familiar  Irrelevant,  novel  relevant,  and  novel  irrelevant  precues,  as 
follows:  An  Item  was  classified  as  either  familiar.  If  Its  precue  was 
factual  and  well-known,  or  novel.  If  its  precue  was  counterfactual.  The 
Item  was  also  classified  as  either  relevant.  If  Its  precue  gave 
informiatlon  helpful  to  Item  solution,  or  irrelevant.  If  Its  precue  was 
unhelpful.  Irrelevant  precues,  edthexigh  unhelpful,  did  oontidn 
infozmatlcmi  assoclatlvely  related  to  the  Item  stem,  so  that  the  precue 
oould  not  be  immediately  recognized  as  Irrelevant. 
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ir.  t-ftgt-g.  Five  psydxmetric  tests  were  administered  to  each 
subject:  the  Verbal  Reasoning  subtest  of  the  Differential  Aptitudes  Test 
(Form  T)  (Bennett,  Seashore,  &  Wesman,  1973} ,  the  cattell  Culture  Fair 
Test  of  g  (Scale  3)  (Cattell  &  Cattell,  1963) ,  the  insist  problems  vised 
by  Sternberg  and  Davidson  (1982) ,  the  Crossing  Out  A*s  Subtest  of  the 
French  Kit  of  Reference  Tests  for  Cognitive  Factors  (French,  Ekstrcn,  & 
Price,  1963) ,  and  the  Vocabulary  Subtest  of  the  French  Kit  (V-3) . 

Design 

Dependent  variables  were  response  times  and  error  rates  for  the 
individual  items.  Independent  variables,  all  within-subjects,  were  test 
type  (anedogies,  classifications,  series  ccopletions) ,  cueing  (vmcued, 
prWxied) ,  and,  for  precued  items,  precue  type:  famil  iarity  (familiar, 
novel) ,  and  relevance  (relevant,  irrelevant) .  Each  itan  had  one  cue 
associated  with  it,  \ihich  was  either  familiar  relevant,  familiar 
irrelevant,  novel  relevant,  or  novel  irrelevant. 

Procedure 


Subjects  first  filled  out  an  informed  consent  form.  Instructions  were 
then  administered  orally.  Subjects  received  the  e9g)eriment2d  tasks 
followed  by  the  psychcnetric  tests.  The  three  experimental  tasks 
(anadogies,  classifications,  and  series  ccopletions)  were  administered  via 
an  Apple  He  mlcroccoputer,  in  counterbalanced  order  across  subjects,  and 
psychcoietric  tests  followed  in  a  fixed  order  in  a  later  session.  Subjects 
were  informed  that  on  precued  items,  the  precue  should  be  accepted  as  true 
and  should  be  applied,  if  appropriate,  to  the  item  that  followed. 

Subjects  were  told  to  respond  as  quickly  and  as  accurately  as  possible. 
Items  within  task  were  blocked  by  precueing  versus  no  precueing,  with  each 
task  consisting  of  four  blocks  of  trials,  ^th  precued  and  vmcued  blocks 
adtemating.  Within  the  precued  condition,  the  four  kinds  of  cues  were 
mixed  so  that  a  subject  could  not  know  on  a  given  triad  what  kind  of  cue 
to  espect.  The  fir^  two  blocks  of  each  ta^  were  each  preceded  by  four 
practice  items.  On  uncued  items,  the  subject  initially  saw  a  blank 
screen;  on  precued  items,  the  subject  initially  saw  the  precue.  When  the 
subject  pressed  RETURN,  the  precue  (if  any)  disappeared,  and  the  test  item 
appeared.  Pressing  of  one  of  four  response  buttons  resulted  in 
t^mlnation  of  the  trial.  Ihe  ocmy^iter  recorded  response  times  and 
responses,  and  then  presented  the  next  item. 

Models  of  Information  Processing 
How  do  subjects  solve  the  kinds  of  factuad  and  oounterfaictuad 
induction  items  presented  in  this  eaperiment?  Such  a  question  can  be 
addressed  by  considering  edtemative  models  of  information  processing.  We 
should  note  that  our  goal  in  this  article  is  not  to  provide  a  relatively 
complete  model  of  information  processing  for  beginniig^-to-end  solution  of 
analogies,  classifications,  and  series  ccopletions:  Such  a  task  has  been 
vndertaken  in  an  earlier  article  (Sternberg  &  Gardner,  1983;  see  edso 
Sternberg,  1985) .  We  have  not  included  the  necessary  task  manipulations 
to  test  the  Stemberg-Gardner  model  cn  the  present  data.  It  is  possible, 
of  course,  that  the  task  manipulations  we  did  iixdude  changed  information 
processing  so  as  to  vitiate  the  applicability  of  our  earlier  model, 
althou^  we  have  no  reason  to  believe  that  this  was  the  case.  Our  goal 
here  is  to  disaws  those  aspects  of  information  processing  that  are 


distinctive  to  the  solution  of  induction  items  with  familiar  or  novel 


precues  that  may  be  either  relevant  or  irrelevant  to  problem  solution. 

Ihree  basic  kinds  of  models  could  account  for  information  processing 
during  solvition  of  the  induction  problems:  the  null  model,  ad^tive 
models,  and  interactive  models.  All  of  the  proposed  models  are  serial, 
we  recognize,  of  course,  that  actual  processing  may  not  be  strictly 
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serial,  and  hence  that  our  models  nay  oversiicpllfy  the  ocniplex  Infcsmatlon 
processing  subjects  actually  did.  But  vie  believe  that  the  models  provide 
at  least  good  approodnatlons  to  possible  subject  strategies.  We  also 
assume  in  all  but  the  null  model  that  processing  of  novel  infomatlon 
takes  longer  than  processing  of  familiar  infomatlon,  in  that  novel 
infomatlon  involves  taiporary  overwriting  of  a  subject's  previous 
knowledge.  As  we  will  show,  this  assunptlon  is  aiiply  si^ported  by  the 
data  in  Table  8.  We  oonsid^  each  of  these  models  in  turn. 

Null  Model 

Model  0.  Model  0  vxxdd  be  the  most  plausible  model  of  information 
processing  If  the  precueing  manipulation  failed.  According  to  this  model, 
the  precueing  manipulation  has  no  effect  unless  the  precue  is  novel 
relevant,  because  only  In  this  condition  is  the  correct  answer  changed  by 
the  information  in  the  precue.  Hence,  It  is  possible  that  the  results 
wculd  show  a  constant  effect  of  the  precue  unless  there  is  actually  seme 
need  of  It  to  compute  a  new  answer.  This  model  predicts  that  novel 
relevant  (NR)  Items  will  be  harder  than  all  the  others,  and  that  these 
others — familiar  relevant  (FR) ,  familiar  irrelevant  (FI) ,  and  novel 
irrelevant  (Nl)--will  be  equal  in  difficulty: 

0.  FR  =  FI  =  NI<NR 
Additive  Models 

Model  1;  Relevant  <  Irrelevant.  In  Model  1,  there  are  separate  time 
charges  for  two  elements  of  infomatlcn  processing:  dealing  with  novelty 
and  deeding  with  Irrelevant  information.  The  motivating  no^on  Is  that 
during  Item  solution,  subjects  are  delayed  by  having  to  take  account  of 
novel  information,  arxi  also  by  having  to  deed  with  irrelevant  information, 
which  Is  recognized  as  irrelevant  only  eifter  a  series  of  successive  f edled 
self-terminating  relevance  tests.  Relevance  is  determined  by  checking 
wheth^  the  oono^itual  relation  in  the  precue  matches  that  in  the  Item  of 
the  problem.  Irrelevance  takes  longer  than  relevance  becaxjse  subjects 
dlsccntlnue  the  relevance  tests  just  as  soon  as  they  see  that  the  precue 
information  Is  relevant  to  solving  the  given  problem.  Rirthermore,  to  the 
extent  that  relevant  precues  act  as  hints,  thi^  may  actually  facilitate 
problem  solution. 

Model  1  can  be  subdivided  into  two  submodels,  la  and  lb.  In  la,  it  is 
presumed  that  the  time  charge  for  novelty  is  greater  than  for 
irrelevance.  In  lb,  it  Is  presumed  that  the  time  charge  for  Irrelevance 
is  greater  than  for  novelty.  In  sum,  the  responseHiime  and  erro3>rate 
predictions  of  Models  la  and  lb  far  the  four  precue  conditions — familiar 
relevant  (FR) ,  familiar  irrelevant  (FI) ,  novel  relevant  (NR) ,  and  novel 
irrelevant  (NI) — ^are: 

la.  FR<FI<NR<NI 

lb.  iR<NR<FI<NI 

Model  2:  Irrelevant  <  Relevant.  Model  2  is  like  Model  1,  except  that 
there  is  an  incremental  time  charge  for  relevance  rather  than  for 
irrelevance.  On  this  model,  novel  informaticn  takes  longer  to  process 
than  familiar  information,  as  in  Model  1.  But  relevant  information  is 
presumed  to  take  longer  to  process  than  irrelevant  Information,  because 
only  relevant  information  needs  to  be  incorporated  into  item  solution.  If 
a  object  determines  information  is  irrelevant,  that  information  can  be 
disregarded.  But  if  the  informaticn  is  relevant,  then  it  is  integrated 
into  the  knowledge  representation  used  to  solve  the  item,  and  this 
integration  plus  the  subsequent  use  of  the  extra  inf orma'U.on  adds  a  time 
charge  to  item  processing,  as  well  as  increasing  the  probability  of  an 
error,  due  to  the  addition  of  an  extra  st^  of  information  processing. 
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Model  2,  lUce  Model  1,  can  be  subdivided  into  two  scQaeodels.  In  Model 
2a,  it  is  presumed  that  the  time  charge  for  novelty  is  greater  than  that 
for  relevance,  vhereas  in  Model  2b,  it  is  presumed  that  the  charge 
for  relevance  is  greater  than  that  for  novelty.  In  sum,  the  response-time 
and  error-rate  predictions  are: 

2a.  FI  <  PR  <  NI  <  NR 

2b.  FI<NI<PR<NR 
Interactive  Models 

Ihe  interactive  models  differ  from  the  additive  models  in  their 
assunption  that  vhether  relevant  or  irrelevant  information  takes  longer  to 
process  depends  vhether  that  information  is  familiar  or  novel. 

Model  3;  Relevant  <  Irrrievanfr  fop  Familiar  Only.  In  Model  3,  as  in 
Models  1  and  2,  there  is  an  incremental  time  charge  for  novelty.  However, 
it  is  assumed  that  for  familiar  items,  successive  selfM:erminating 
searches  result  in  irrelevant  information  being  slower  to  incorporate  than 
is  relevant  information  (as  in  Model  1) .  Moreover,  because  familiar 
relevant  Information  does  not  change  the  keyed  response  on  familiar  items, 
such  information  can  immediately  be  discarded  withi^  further  processing. 
On  novel  relevant  items,  however,  relevant  information  does  ohange  the 
]oeyed  answer,  and  hence  must  be  incorporated  into  solution  processing, 
resulting  in  an  added  tine  charge  for  relevant  information  in  items  idth 
novel  precues. 

3.  PR<FI<NI<NR 

Model  4;  Relevant  <  Irreltyant  for  Novel  Only.  In  Model  4,  as  in  the 
preceding  models,  there  is  an  incremental  tine  charge  for  novelty. 

However,  it  is  assumed  that  on  items  with  familiar  precues,  ino^porating 
precue  information  into  item  solution  results  in  these  items  taki^  longer 
to  solve  than  items  with  familiar  irrelevant  precues  (as  in  Model  2) .  On 
items  with  novel  precues,  it  is  assumed  that  irrelevant  information  takes 
longer  to  process,  because  it  is  more  difficult  to  recognize  the  precue 
information  as  irrelevant.  In  other  words,  whereas  the  relevance  of 
familiar  information  can  be  readily  assessed,  the  relevance  of  information 
is  hard  to  assess  vhen  the  information  is  novel  in  the  first  place;  hence, 
the  delay  for  irrelevant  information  when  processed  in  the  context  of  a 
novel  precue. 

4.  PT<PR<NR<NI 

Results 

Reliability  of  Data 

A  fundament2d,  preliminary  issue  that  needs  to  be  addressed  before 
subsequent  data  an£dysis  is  whether  the  data  for  the  experimental  ta;^  are 
reliable.  Ihe  task  is  a  rather  unusual  one,  and  it  is  possible  that,  as  a 
result,  the  data  will  lack  internal  consistency,  either  with  respect  to 
items  (each  item  measuring  a  different  ability)  or  with  re^)ect  to 
subjects  (each  subject  eoploying  a  different  strategy) .  Item 
coefficient-alpha  reliabilities  are  equivalent  to  reliabilities  for  all 
possible  split  halves  of  items,  \ihereas  subject  coefficient-alpha 
reliabilities  are  equivalent  to  reliabilities  for  all  possible  split 
halves  of  subjects.  Ihese  intemad-conslstency  reliabilities  are  shown  in 
Table  7.  All  of  the  reliabilities  are  in  the  high  .80s  and  low  .90s, 
vhich  are  hi^Uy  satisfactory  for  data-analytic  purposes. 


insert  I^le  7  about  here 


OorrelatlGDs  between.  Response  Tiroes  and  Error  Rates 

A  second  fundamental  and  preliminary  issue  concerns  correlations 
between  response  times  and  error  rates.  This  correlation  was  .62  (p  < 
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.001)  across  subjects,  indicating  that  there  was  no  ^)eed-aocuracy 
tradeoff,  but  rather,  that  subjects  uho  were  faster  were  also  more 
accurate.  Ihe  oorrelaticn  was  .21  (p  <  .001)  across  items.  Indicating 
tiiat  items  that  took  longer  to  solve  were  also  more  susceptible  to  error. 
In  sum,  we  need  oonoem  ourselves  no  further  either  with  ^>eed-aocuracy 
tradeoff  nor  with  the  possibility  that  errors  tended  to  be  "quick"  ones 
due  to  subjects  giving  vqp  without  fUUy  atteopting  item  solution. 

Basic  Statistics 

Response  arid  error  rates.  Basic  statistics  for  the  various 

e9<perimBnt2d.  conditions  are  shown  in  Table  8.  Means  are  shown  for  both 
correct  response  times  and  error  rates  as  a  function  of  item  type 
(anzdogles,  classifications,  series) ,  cueing  condition  (cued  or  uncued) , 
and  type  of  precue  information  (familiar  relevant,  familiar  irrelevant, 
novel  relevant,  novel  Irrelevant) .  (Patterns  of  results  are  essentially 
identical  if  all  response  times,  including  error  ones,  are  analyzed.) 


Insert  Table  8  about  here 


Table  9  shows  an  an2Q.ysls  of  variance  upon  these  means.  Ihe  results 
are  clearcut:  All  of  the  main  effects  and  interactions  are  statistically 
significant  for  the  re^nnse  tines,  and  edJ.  of  the  main  effects  and  all 
but  one  of  the  interactions  are  statistically  significant  for  error 
rates.  These  results  indicate  that  the  esp^dmental  memipulations 
affected  performance,  but  more  importantly,  that  these  effects  were 
interactive,  as  would  be  predicted  from  either  Mcxiel  3  or  4.  For  the  main 
effects,  series  oonpletiora  take  longer  to  solve  than  classifications, 
which  in  turn  take  longer  to  solve  than  analogies.  Items  with  novel 
precu^  take  longer  to  solve  than  items  with  familiar  precues,  and 
irrelevant  precues  take  longer  than  relevant  ones.  ANO\^  were  2aso 
cedculated  for  each  task,  and  they  were  essentially  the  same  as  the 
overall  AMCfVAs. 


Insert  Table  9  about  here 


The  various  medn  effects  and  interactions  can  be  better  vnderstood  in 
the  context  of  Table  8,  whioh  includes  mean  reqanse  times  and  error  rates 
on  the  various  item  '^pes  for  each  of  the  four  conditions  of  precueing. 
Reaction  times  and  error  rates  show  identical  patterns:  Overall  means,  as 
well  as  the  means  for  the  analogy  and  classifications  tasks,  clearly 
support  Mcxiel  3,  the  interactive  model  in  which  irrelevance  exkis  to 
re^nnse  times  and  error  rates  for  items  with  familiar  precues,  but  in 
which  relevance  adds  to  re^xsnse  times  and  error  rates  for  items  with 
novel  precues.  In  the  series  ccopletions,  the  re^ense  times  and  error 
rates  for  novel  relevant  items  are  relatively  low,  resulting  in  these 
means  sipporting  Mcxiel  lb  rather  than  Model  3.  Thus,  the  individued  task 
results  and  the  overall  means  for  re^xsnse  times  and  error  rates  sipport 
Model  3,  except  for  the  series  ocnple^ons  task,  vhere  the  results  sipport 
Mcxiel  lb. 

So  far  in  our  models  we  have  only  considered  sirple  scores.  It  is 
also  informative,  however,  to  look  at  the  pattern  of  differences  between 
cued  and  uncued  scores  in  order  to  examine  the  effect  of  cues  on  prxablem 
solution  (Table  10) .  The  expected,  obvious  patterns  occur  clearly:  that 
the  difference  scores  for  items  with  Irrelevant  precues  anVor  novel 
precues  are  all  positive;  that  is,  both  novel  and  Irrelevant  precued  items 


CNR  Ooertract  N0001485K0589  Finad  depart,  Page  14 


h2tve  a  higher  irespcnse  time  and  onnor  rata  than  the  sane  Items  iincued.  It 
is  most  interesting,  hcMever,  to  look  at  the  scores  for  items  having 
familiar  relevant  precues,  becaxise  s^viticn  of  these  items  is  sanetimes 
hindered  and  sometimes  fai^itated  hy  the  presence  of  precues. 


Insert  Table  10  about  here 


Ihe  familiar  relevant  precues  make  analogies  more  difficult  vftiile 
feicilitating  solxzticn  of  classifications  and  series  coopletions.  There 
are  two  possible  reasons  for  this  effect.  The  first  reason  relates  to  the 
difficulty  of  the  type  of  item.  As  can  readily  be  seen  from  Table  8, 
analogies  are  clearly  the  easiest  uncued  items.  This  is  probably  because 
solving  analogies  is  a  well-practiced  skill  in  the  population  sanpled 
(Yede  undergraduates) .  The  analogies  in  this  eogseriment,  vftiioh  employ 
only  onmm.  words  and  concepts,  can  be  solved  almost  automatically  by 
these  subjects,  whereas  the  classifications  and  series  completions  are 
less  familiar  and  hence  more  difficult.  The  effect  of  a  precue — even  a 
familiar  relevant  one --on  analogy  solving  is  to  cause  an  interruption  of 
processes  that  would  otherwise  proooed  automatlcedly.  Even  if  the  precue 
gives  helpful  information,  incorporating  it  espends  more  time  than  it 
saves.  On  the  other  hand,  the  precues  for  clzissiflcations  and  series 
ocmpletlons  are  potentially  more  helpful  because  the  subjects  need  more 
help  on  the  more  difficult  items. 

The  second  possible  reason  for  the  differential  helpfulness  of 
familiar  relevant  cues  is  that  the  more  flexible  the  structure  of  an  item, 
the  more  difficult  it  is  to  determine  the  relevance  of  a  precue. 

Analogies  are  hi^y  flexible  in  that  there  can  be  a  wide  variety  of 
relations  between  terms,  consider,  for  example,  the  relationships  between 
BAGEL  and  DOUGHNUT,  between  HERO  and  ADMJRATICN,  and  between  EYE  and 
BUNDING  (all  from  test  items) .  In  classifications,  the  relations  between 
terms  are  more  constrained;  in  an  item,  2dl  terms  eune  the  same  part  of 
speocfa,  and  they  are  all  related  by  grcup  sembership.  In  series  problems, 
the  relationships  between  terms  are  even  further  restricted;  in  ai^tion 
to  satisfying  the  constraints  mentioned  above  for  classifications,  the 
terms  must  also  be  arranged  in  a  progression.  Thus,  in  analogies  it 
should  be  fairly  difficult  to  deteosnine  whether  or  not  a  precue  contains 
relevant  information.  In  other  words,  nearly  any  precue  which  mentions 
any  aspect  of  any  of  the  analogy  terms  could  conceivably  be  relevant  to 
problem  solvitlon. 

In  classifications,  because  the  terms  must  be  related  by  groip 
msobership,  it  should  be  fedrly  easy  to  spot  a  relevant  precue,  and  a 
relevant  precue  should  be  fairly  helpful,  in  that  it  gives  away  the  rule 
of  to  what  grocp  the  terms  belong.  The  precue  in  series  problans  is  even 
easier  to  recopilze  as  relevant  and  is  more  helpful,  in  that  if  it  is 
relevant,  it  really  "gives  away"  the  answer,  and  if  not,  it  is  readily 
perceived  as  unhelpful,  and  can  be  quickly  discarded. 

According  to  the  above  rationale,  the  following  would  be  expected: 

For  analogies,  both  relevant  and  Irr^evant  cues  are  xmhelpful  and 
tiiae-CGnsuming,  in  that  they  mainly  distract,  whereas  for  classifications 
and  series  ccnpletions,  relevant  precues  are  helpful,  and  it  is  less 
difficult  to  recognize  a  precue  as  relevant.  These  expectations  do 
describe  the  pattern  of  results  obtained  for  items  with  familiar  precues. 
As  shown  by  both  re^xnse  times  and  error  rates,  solution  of  analogies  is 
inpeded  by  precues,  whereas  solution  of  classifications  and  series 
ccnpletions  is  facilitated  by  precues. 


OMR  Ocntxact  N0001485K0589  Firal  Report,  Pag&  15 


Similar  patterns  hold  for  the  novel  precxies  for  analogies  and  series 
ccnpletions;  however,  the  results  for  novel  classification  are 
ancnalous.  Ihe  reason  seems  to  relate  to  the  actual  novel  relevant 
used  in  the  classifications.  The  most  obvious  ^pe  of  novel  relevant 
precue  which  could  hatve  been  constructed  for  classification  problems 
turned  out  to  be  trivially  ea;^,  such  as: 

Celery  is  a  fruit. 

Sjpple,  pear,  peach,  grapes, 

1.  orchard 

2.  produce 

3.  celery 

4.  jam 

In  order  to  avoid  such  Items,  which  merely  entail  substitution  rather  than 
reasoning,  the  Items  were  constructed  as  follows: 

Fruits  have  long  green  stadJcs. 

apple,  pear,  peach,  grapes, 

1.  orchard 

2.  produce 

3.  celery 

4.  jam 

Itaus,  in  the  novel  precues  for  classifications  (unllloe  those  for 
analogies  and  series  ocnpletlons) ,  the  change  induced  by  the  precue  Is  a 
change  in  the  properties  of  the  words  in  the  stem,  not  in  the  word  vhlch 
oonpletes  the  Item.  Ihe  effect  of  this  extra  inductive  step  was  to  make 
the  novel  relevant  precues  more  difficult  to  reason  with  for 
classifications  than  for  either  analogies  or  series  ccnpletions. 
Oonseguently,  the  novel  relevant  classifications  have  ccnparatively  high 
reaction  times  and  error  rates. 

In  sunnary,  the  effect  of  precues  seems  to  be  as  follows:  Precues 
hinder  Item  solution  If  the  precue  Is  novel  anVor  Irrelevant,  If  the  Item 
is  easy  and  automatized  in  the  uncued  state.  If  the  structure  of  the  Item 
is  flexible  encu^  that  It  is  difficult  to  determine  whether  or  not  the 
precue  Is  relevant,  or  If  the  precue  Is  nonobvlous  by  being  several 
inductive  steps  away  from  the  problem  stem.  Conversely,  a  precue  can 
facilitate  Item  solution  If  the  precue  is  familiar  relevant,  the  item  Is 
fairly  difficult  to  start  with,  the  Item  Is  of  a  rigidly  stmctured  type 
vhere  the  precue  can  be  readily  recognized  as  helpful,  and  the  precue  Is 
inductively  close  to  the  Item  stem. 


Table  11  shcMs  oorrelatlcns  between  esperimental  task  and  psychcmetric 
test  scores.  Correlations  are  shown  both  for  the  Individual  tests,  and 
for  approKimation  factor  scores  based  on  a  varimax-rotated  princip^-axis 
factor  analysis  of  tlie  data.  The  factor  anedysls  yielded  two 
interpretable  factors  with  elgenvalvies  greater  than  1,  namely,  a  reasoning 
factor  (CAT  Vedaal  Raasoning,  Cattell  Culture-Fair  Test  of  g,  Insl^it 
Test)  and  a  verbal/^arceptual  factor  (Crossing  Out  A's,  Vocabulary) . 
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lunnebarg,  1973,  for  similar  logic) .  Seccnd,  do  the  correlations  vdth  the 
reasoning  tests  (and  factor)  differ  frcn  those  with  the  verbal/^ieroeptual 
factor?  In  particular,  the  reasoning  tests  were  chosen  to  measure  fluid 
abilities  similar  to  those  tapped  hy  the  eogserimental  tasJcs,  whereas  the 
verbal  and  perceptual  tasks  were  chosen  to  measure  crystallized  abilities 
different  fran  those  temped  by  the  eaperlmenttol  tasks  (see  Hunt, 

Lumdsorg,  &  Lewis,  1975,  for  an  informatlcn-prooessing  analysis  of 
verbed,  or  cxystedllzed  abilities) .  Ihlrd,  are  the  correlations  for  the 
uncued  and  cued  conditions  with  the  psychcDetric  tests  different?  The 
trlarchlc  theory  predicts  that  the  more  nonentrenched  (precued)  Items 
should  be  the  better  measures  of  intelligence,  and  the  peychcmetric  tests 
of  flxiid  ability  were  ohoeen  for  being  among  the  best  measures  of  fluid 
reasoning  ability.  Hence,  the  corxelatlcns  for  the  precued  Items  might  be 
expected  to  be  higher  than  those  for  the  uncued  Items. 

The  answers  to  these  questions  are  fedrly  strzd^itforward.  First, 
most  of  the  correlations  in  the  table  are  statistically  significant,  and 
generally  low  to  moderate  in  magnitude.  Second,  correlations  of  the 
experimental  tasks  with  the  reasoning  tasks  are  clearly  hl^ier  than  those 
with  the  verbal/peroeptual  tasks.  For  the  factor  scores,  for  example,  all 
of  the  correlations  of  reaction  times  and  of  error  rates  with  the 
reasoning  factor  (ranging  from  -.31  to  -.46)  are  statistically 
significant,  vhereas  none  of  the  correlations  with  the  verbal/peroeptual 
factor  (ranging  from  -.05  to  -.17)  are  significant.  Thus,  the 
ejperimental  tasks  do  appear  to  t^  abilities  related  to  those  tapped  by 
the  psychcmetrlc  tests.  Third,  six  of  six  ocnparisons  between 
correlations  of  uncued  versus  precued  response  times  and  error  rates  with 
the  peychanetrlc  reasoning  tests  come  out  hi^ner  for  the  precued  than  for 
the  uncued  items.  In  sum,  the  ejperimental  tasks  showed  the  predicted 
patterns  of  convergent-discriminant  vsaidlty  with  re^sect  to  the 
psychometric  tests. 

Discussion 

This  experiment  investigated  information  processing  during  the 
solution  of  analogies,  classifications,  and  series  ocnpletlons  either 
taking  the  standard  form  or  else  preceded  by  precueing  information  that 
could  be  novel  or  familiar,  and  relevant  or  Irrelevant.  The  data  analyses 
investigated  both  the  intexml  and  external  validities  of  the  tasks. 

With  reflect  to  Internal  validation.  It  was  found  that  for  analogies 
and  classifications,  subjects  take  longer  to  process  irrelevant  than 
relevant  information  If  the  precue  Is  familiar,  but  they  take  longer  to 
process  relevant  than  Irrelevant  information  If  the  precue  Is  novel.  For 
series  completions,  however,  both  novelty  and  Irrelevance  add  time  to  the 
processing  of  Informatlcr,  with  the  time  for  irrelevance  greater  than  that 
for  novelty.  Figure  1  shews  a  genered  information-processing 
model — formalized  as  a  flow  chart — ^that  provides  a  plausible  account  of 
information  processing  in  edl  three  tasks.  The  tasks  differ  only  in  the 
proportion  of  time  the  various  steps  take,  which  Is  a  factor  of  Item  type, 
cue  difficulty,  and  so  on.  These  variables  eiffect  vhat  ^seclflc 
Information-processing  model  spiles  to  a  given  task.  For  instance,  as 
previously  discussed,  applying  novel  relevant  precues  Is  easy  for  series 
ccnpletlons  relative  to  the  other  two  tasks,  so  the  resulting  model  is 
additive  rather  than  interactive. 


Insert  Figure  1  about  here 


In  the  general  model  shewn  in  the  flew  chart  (which  applies  to  precued 
Items  only — for  standard  models,  see  Sternberg  &  Gardner,  1983) ,  sc£>jects 
first  read  the  precue.  If  the  precue  is  familiar,  then  the  subjects 
access  category  Information  needed  for  problem  solution.  Subjects  then 
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self-'tenninating  as  soon  as  they  find  hcv  the  infomaticn  is  relevant  to 
item  solution.  At  this  point,  th^  incorporate  the  precue  if  it  is 
relevant,  solve  the  problem,  and  then  respond  (see  Sternberg  &  Gardner, 
1983,  for  hew  similar  uncued  problems  are  solved) .  If  successive  tests 
f2dl  to  shew  the  relevance  of  the  precue,  subjects  discontinue  relevance 
checking,  reject  the  precue  as  irr^evant,  solve  the  problem,  and  respond. 

If  the  precue  is  novel  rather  than  familiar,  then  subjects  have  to 
create  nev  category  information  far  the  oounterf actual.  After  requesting 
the  item  stem,  they  then  do  a  series  of  relevance  chedcs  for  the  novel 
precue.  If,  after  a  series  of  cd^cks,  they  axe  unable  to  see  any 
relevance  for  the  precue,  they  reject  the  cue,  solve  the  problem,  and 
respond.  If,  hewever,  the  precue  is  relevant,  then  they  ociDbine  the 
precue  with  the  stem,  a  nontxivial  process  since  the  precue  not  only 
overrides  their  previous  knowledge  but  also  changes  the  answer  to  the 
problem.  After  oenbining  the  cue  with  the  stem,  the  subjects  solve  the 
problem  and  respond. 

With  respect  to  external  validation,  it  was  found  that  the 
nonentrenched  induction  tasks  overlapped  with  psychcmetric  tests  in  terns 
of  abilities  measured,  that  the  abilities  measured  were  fluid  rather  than 
crystallized,  and  that  the  precued  (more  nonentrenched)  items  were  better 
measures  of  fluid  abilities  than  were  the  tnxued  ones.  Ihe  nonentrenched 
induction  tasks  thus  fulfill  their  original  goal  of  being  essentially 
IQ-^test-like  in  their  surface  structure,  in  that  they  can  be  answered 
rather  quickly  with  an  unanbiguous  keyed  answer.  But  they  are  unlike 
standard  items  on  intelligence  tests  in  being  more  nonentrenched— or 
unusual— in  their  infornation-prooessing  requirements. 

The  results  of  this  esqieriment  are  consistent  with  the  notions  of 
Raaheim  (1974),  Snew  (1980),  and  Sternberg  (1982,  1985)  that  relatively 
nonentrenched  tasks  provide  particularly  apt  measures  of  intelligence,  and 
particularly,  of  flxild  intelligence.  The  ability  to  cope  with  relative 
novelty  is  an  important  aspect  of  intelligence,  axxi  it  can  be  meaisured 
efficiently  and  in  a  theoretically-based  way  using  the  nonentrenched 
analogies,  classifications,  and  series  ocnpletions  used  in  the  present 
research. 


E}q}ertise  and  Flexibility:  The  Costs  of  Expertise^ 

During  the  past  three  decades,  research  efforts  directed  toward 
understanding  the  nature  of  eoq^ertise  have  increased  tremendously. 
Differences  between  ejq^rts  and  noneo^erts  in  ccsplex  problem-solving 
domains  have  been  shown  to  be  both  quantitative  and  qualitative.  Not  only 
do  experts  perform  better  than  do  nonesq^rts  on  quantitative  measures,  but 
they  also  seem  to  structure  problem  representations  differently  and  appear 
to  eqsply  different  strategies.  Across  a  variety  of  domains,  it  is  found 
that  experts  tend  to  oonoeptualize  dcoain-related  problems  in  abstract 
terms,  vbereas  nonmtperts  apparently  rely  more  on  surface-level  features. 

For  exanple,  in  their  classic  stucy  on  expert  chess  players.  Chase  and 
Simton  (1973) ,  replicating  deC^xxst's  (1965)  earlier  findiigs,  re^xarted  that 
more  eaqjerlenced  players  spent  less  time,  made  fewer  errors,  needed  fewer 
glances,  and  took  in  more  information  per  glance  than  did  less  eaqjerienced 
players  when  their  task  consisted  of  memorizing  or  rqaroduclng  briefly 
presented,  meaningful  chess  patterns.  Hewever,  esqierts'  encoding  ability 
dropped  to  the  level  of  a  no^ce  whmi  the  expei±s  were  forced  to  dAa]  vdth 
meaningless  chess  patterns.  Examining  the  recall  cl\;isters  of  their 
players.  Chase  and  Simon  found  that  the  clusters  of  egaerlenoed  players 
frequently  were  based  upon  attack  or  defense  configurations,  implying  an 
abstract  knowledge  representation. 
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Chi,  Glaser,  and  Rees  (1981)  extended  this  result  to  pt^lclsts  on  the 
basis  of  ccncepibial  sorting  tasks  and  verbal  protocols.  Ihey  suggested 
that  eag^erts'  problem  representations  tend  to  focus  on  abstract  physical 
principles,  whereas  novices  represent  probleois  in  terns  of  surface 
features.  Researchers  investigating  different  skill  dcnains  have  arrived 
consistently  at  the  same  conclusion  (e.g. ,  Adelson,  1981;  1984;  Chamess, 
1979;  Kay  &  Black,  1985;  Reitman,  1976;  Sohoenfeld  &  Hermann,  1982) . 

Studies  focusing  on  the  structure  of  knovledge  representations 
generally  atteopt  to  understand  (^jaditative  ohanges  that  arise  with 
growing  esgiertise  in  a  field.  Ihese  studies,  however,  do  not  necessarily 
address  the  issue  of  quantitative  changes  in  the  cognitive  system  that 
2lLso  go  edong  with  increasing  e^qsertise  and  that  leekd  to  remarkable 
decreases  in  qpeed  of  performanoe.  For  exaitple,  in  a  stuc^  of  expert  and 
zianeaepert  bri^e  players,  Chamess  (1979)  showed  that  esg^erts  needed  less 
time  to  perform  qualitatively  better  than  did  novices  in  four  different, 
bridge-related  tasks:  planning  the  play  of  a  contract,  rapid  bidding, 
incidental  learning,  and  recall  of  briefly  exposed,  meaningful  brid^ 
hands.  The  only  task  in  which  esperts  did  not  perfom  faster  and  better 
than  did  nonexperts  was  the  recall  of  ]»lefly  presented,  voistructured 
hands. 

Several  researchers  have  proposed  theories  that  address  why  experts 
can  solve  problems  more  quickly  than  ncneaperts  can  (e.g.,  Cheng,  1985; 
Kahneman,  1973) .  One  of  the  most  elaborated  among  those  theories  was 
introduced  by  Anderson  (1982) .  Based  upon  the  mechanisms  of  his  ACT* 
model  of  oo^itive  architecture,  Anderson  proposed  that  expertise  develops 
from  an  earlier  declarative  stai^,  in  vhich  general  inter^nretative 
procedures  are  applied  to  facts  about  the  skill  dcnain,  to  a  procedural 
stage  in  which  the  domain  knowledge  is  oonpiled  into  procedures. 

According  to  his  view,  the  mastery  of  a  cognitive  skill  is  never 
ocnpletely  finished.  Even  cccplled  procedures  continue  to  be  fine-tuned 
by  means  of  gaieralizatlon,  discrimination,  and  strengthening. 

Taken  together,  the  above  findings  suggest  that  the  degree  of 
proceduralization  and  the  abstract  nature  of  the  knowledge  organization 
are  among  the  most  isportant  ingredients  of  skilled  problem-solving. 
Proceduralized  danainrq»cific  problem-solving  strategies  operating  on 
problem-suited  knowledge  representations  give  eoperts  both  a  ^»ed  and  a 
quality  advantage  over  nonexperts. 

Generally,  studies  on  eipertise  have  focused  on  understanding  \hy 
experts  do  better  than  nonesperts.  Theories  of  espertlse,  such  as  the 
ones  described  above,  have  been  constructed  to  account  for  experts' 
scperior  problem-solving  ability,  thereby  esnphaslzing  the  benefits  of 
expertise.  Such  theories,  however,  should  not  be  restricted  to  situaticns 
in  which  experts  outyerform  nonexperts.  Gcoplete  theories  of  expertise 
should  be  able  to  predict  under  vhich  circunietarvoes  experts  will  not 
perform  better  than  noneoperts  or  will  even  be  outperformed  by 
rxnesperts.  So  far,  however,  theories  have  only  occasionally  been  linked 
to  the  costs  of  e>pertise  (bixt  see  Adelson,  1984) . 

One  reason  why  experts  sometimes  mi^it  be  outperformed  by  nonesperts 
could  derive  from  the  inflexibility  of  their  information-processing 
systan.  Vte  hypothesize  that  a  cogitative  system  that  has  been  developed 
specifically  to  perform  within  the  boundaries  of  a  well-i^»clfied  problem 
area  need  not  necessarily  be  very  flexible.  This  inflexibility  ml^t 
reveal  Itself  in  one  or  both  of  two  different  ways.  First,  the  system 
could  produce  negative  transfer  as  a  result  of  being  unable  to  inhibit  old 
solution  strategies  vhen  response  requirements  change.  Thus,  an  expert 
physicist,  for  exanple,  mi^it  find  it  very  difficult  to  inhibit  older. 
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prooedurallzed  e}q>lanatlcn  patterns  to  account  for  familiar  problems  even 
though  she  is  aMare  of  nev  findings  that  indicate  the  in2ideguacy  of  such 
solutions.  Second,  the  cognitive  ^stem's  knowledge  base  mi^  be  hard  to 
modify.  To  stay  with  our  exasple:  Even  if  our  physicist  h2Ki  no  problems 
Inhibiting  old  solutions,  she  mic^  still  be  hard  pressed  to  incorporate 
the  new  findings  into  her  existing  loxiwledge  base. 

Inhibition  and  adaptation,  the  two  oonponents  of  flexibility,  have 
both  been  used  as  explanatory  mechanisms  in  the  long  oontroversy 
surrounding  theories  of  associative  learning.  Different  frameworks 
expladning  the  negative  interference  effects  obtadned  in  the  basic  A-B, 

A-C  paradigm  have  tended  to  eophasize  one  or  the  other  of  the  two 
different  aspects  of  flexibility.  Khereas  proactive  interference 
(e.g.  ,Uhderwood,  1957)  and  list  discrimination  accounts  (e.g. ,  Sternberg  & 
Bower,  1974)  desoribed  interference  effects  in  terms  of  subjects' 
inability  to  inhibit  older,  previously  established  associations, 
organizational  theory  (e.g. ,  Tulving,  1966)  tried  to  link  them  to 
adaptation  difficulties. 

In  an  attenpt  to  xaiderstand  why  people  find  it  difficult  to  reason 
with  new  conc^xts,  Tetewsky  and  Sternberg  (1986)  showed  that  subjects 
experience  more  dif f icultiitt  in  novel  situations  when  concepts  are  partly 
unfamiliar  than  when  they  are  ccnpletely  unfamiliar.  Tetewsky  and 
Sternberg  infer  to  adaptation  difficulty  and  iiihibition  difficulty  as  two 
possible  explanations  for  the  result  thty  obt2dned. 

We  claim  that  inflexibility  and  exp^tise  are  inexctrlcably  linked.  A 
problem  solver's  inability  to  inhibit  exdsting  response  patterns  and  to 
modify  old  knowledge  when  response  requirements  change  (i.e. , 
infl&doility)  are  affected  by  his  knowledge  arganlzation  and  by  its 
degree  of  prooeduredization  (i.e. ,  expertise) .  A  hi^y  structured  system 
that  has  prooeduralized  its  processing  of  knowledge  will  be  relatively 
inflexible. 

In  Experiment  1  we  test  the  hypotheses  that  (a)  the  structure  of  a 
knowledge  base  and  (b)  its  degree  of  prooedurell^tlon  eiffect  the 
flexibility  with  which  a  problem  solver  can  respond  to  novel  task 
demands.  We  expect  that  problem  solve»  who  have  proceduredlzed  their 
solution  strategies  to  a  great  extent  will  be  more  effected  by  changes  in 
task  demands  than  will  problem  solvers  vho  respond  to  problems  in  a  less 
procedurallzed  manner.  Ihus,  experts  in  any  given  dcnedn  will  generally 
be  more  vulnerable  to  task  demand  modifications  than  nonexperts  will  be. 
Furthermore,  we  assume  that  the  effects  of  changes  in  task  are 

larger  for  those  demands  that  are  inocnpatible  i^th  the  structure  of  the 
knowledge  base  than  for  demands  that  are  ompatible.  Therefore,  expex^s 
will  be  more  affected  when  new  task  demands  csdl  for  deep,  abstract 
principles  to  be  changed  than  when  surface  features  are  to  be  changed. 

For  nonexperts,  the  reverse  is  expected.  That  is,  they  will  be  more 
eiffected  by  surface  changes  than  by  deep,  abstract  changes. 

The  specific  domain  of  knowledge  chosen  to  test  these  ideas  is  the 
game  of  bridge,  (ippendix  A  describes  the  basic  bridge  rules  for  readers 
unfamiliar  with  the  game. )  There  have  been  only  few  studies  on  skill 
differences  in  bridge  (e.g.,  Chamess,  1979;  Engle  &  Bukstel,  1978).  The 
few  studies  conducted  generally  obtained  results  that  are  compatible  with 
the  findings  reported  in  other  dccains.  It  appears  that  master  bridge 
players  encode  abstract  distributional  features  of  bridge  hands  that 
automaticedly  evoke  strategies  containing  long  lines  of  play,  like  "play 
an  end  game."  Nonexperts,  on  the  other  hand,  seem  to  encode  hands 
primarily  in  terms  of  surface  features  (honor-cards,  for  instance) ,  that 
axre  associated  with  smaller  lines  of  play,  li3oe  "take  a  trick." 
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In  the  first  experiment,  bridge  players  of  vzurylng  levels  of  skill 
played  12  simulated  bridge  games  on  a  ocoputer.  Half  of  the  games  were 
played  under  nomnal  conditions.  In  the  other  half,  players  were 
insbnxted  to  play  slic^itly  diff^nent  versions  of  bridge.  Version  1 
introduced  new  nonsense  names  for  honoi>cards  and  suits;  version  2 
rearraii^  the  order  of  honor-cards  and  suits;  and  version  3,  the 
lead-rule  change,  modified  the  rule  determining  vho  began  each  play. 
Instead  of  the  player  vho  won  the  last  trick,  vhich  is  the  i-rwim  rule  in 
bridge,  the  player  with  the  lowest  card  in  the  last  trick  led  into  the 
next  trick.  The  different  versions  were  intended  to  tap  different  levels 
of  subjects'  information  processing.  Versions  1  and  2  were  considered 
surface  modifications;  version  3  was  expected  to  exert  its  effect  on  a 
deeper,  more  abstract  and  strategic  level. 

E>q)erimMit  1 

Bgieriment  1  tested  three  different  hypotheses.  First,  ei^erts  will 
play  bridge  more  quickly  and  qualitatively  better  than  nones^jerts  will 
vhen  the  origined  bridge  rules  are  in  effect.  Second,  esqserts  will  be 
more  affected  by  the  three  rule  modifications  than  nonei^ierts  will  be. 
Finedly,  e}q)erts  will  be  more  eiffected  by  the  doq>-structural  change  than 
by  the  surface-structural  changes,  vhereas  for  noneaqserts,  the  reverse 
will  be  true. 

Subjects 

Thirty-four  bridge  players  (16  itales  aixi  18  females)  from  the  New 
Haven  area  ware  recruited  via  £xivertis«anenta  in  three  local  duplicate 
bridge  clubs.  The  president  of  each  cluib  assigned  each  player  to  cne  of 
10  categories  according  to  level  of  skill.  The  category  Eystem  that  was 
used  is  shewn  in  i^pen^  B.  Expertise  scores  for  each  subject  were 
obtained  by  averaging  across  the  three  raters'  assignments.  The  players 
ranged  in  age  from  21  to  70.  They  were  run  in  individual  sessions  and 
were  pedd  $5  per  hour  for  their  participation. 

Materials 

Experimental  task.  The  bridge  program  used  in  this  stuc^  was  based  cxi 
Borland  International's  Turbo  Pascal  version  of  Hirbo  Bridge  and  was 
modified  by  Peter  Frensch  to  suit  the  purposes  of  the  stuefy.  It  was  run 
on  an  lEM  PC  microoenputer. 

Hand  distributions  of  all  games  were  randomly  chosen  by  Peter  Frensch 
prior  to  data  collection  (out  of  a  pool  of  oenputar^-generated  random 
games)  and  were  displayed  on  the  screen  using  the  saone  abstract 
representation  as  is  ocmmonly  found  in  bridge  books  and  newspaper 
columns.  The  screen  dl^lay  was  identical  for  the  bidding  and  playing 
phases  of  a  game.  Figure  2  shows  a  hypothetical  screen  configuration  as 
it  might  have  eppeared  in  the  first  trick  of  a  game. 
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program's  activities  was  continuously  dij^layed  (e.g. ,  "North  bidding," 
"West  playing") .  Subjects  were  proipted  to  make  a  bid  or  to  select  a  card 
(e.g. ,  "South:  ?") .  Card  and  suit  ranks  (bid  levels  and  bid  suits  during 
the  bidding  phase)  were  displayed  together  with  the  prompt.  Stpeciail  keys 
on  the  oosputer  keyboard  were  assigned  to  cards,  bid  levels,  and  suits. 

Three  different  modifications  of  the  regular  bridge  rules  were  \;ised:  a 
name  change,  a  rank-order  change,  and  a  lead  change.  In  the  name-change 
condition,  the  familiar  card  and  suit  names  (ace,  king,  queen,  jack,  and 
spade,  heart,  diamond,  club)  were  replaced  by  nonfamiliar,  nonsense  names 
(rutz,  lark,  dill,  beib  for  cards,  and  pular,  biref,  ramog,  kamer  for 
suits,  respectively) .  The  rank-order  change  modified  the  familiar  card 
and  suit  rank  orders  (aoe,  king,  queen,  jack  to  king,  jack,  ace,  queen, 
respectively,  for  cards,  and  spade,  heart,  diamond,  cl\;ib  to  diamond, 
heart,  cl\;ib,  spade,  resE»ctively,  for  suits) .  The  lead  change  modified 
vho  began  each  play.  Instead  of  the  player  vho  won  the  last  trick,  the 
player  with  the  lowest  card  in  the  last  trick  led  into  the  next  trick. 

Qaestionnaire.  In  addition  to  participating  in  the  eaperimartal  task, 
subjects  were  given  a  questionnadre  asking  their  age,  nunher  of  yeeuns  th^ 
hacve  played  bridge,  nunher  of  master  points  accumulated  in  total  and 
during  the  past  two  years,  frequency  of  dtplicate  and  rubber  bridge  play 
per  month,  an  estimate  of  their  level  of  bridge  skill  on  a  1-9  scale  (from 
poor  to  excellent) ,  and  their  eoperienoe  with  other  card  games.  (>hster 
points  in  bridge  are  awarded  by  the  American  Contract  Bridge  league  to 
those  scoring  in  the  top  positions  at  dipllcate  bridge  toumaments.  The 
number  of  points  varies  with  the  number  of  participants  and  type  of 
event.) 

Design 

The  dependent  variables  were  play  time,  number  of  tricks  won  per  game, 
and  number  of  games  won.  Independent  bebween-subjects  variables  were 
level  of  e9q»rtlse  (nonejpert,  expert)  and  type  of  rule  modification  (name 
change,  rank-order  change,  lead  change) .  Game  (1  to  12)  «as  a 
within-subjects  variable.  In  each  espertise  groip,  subjects  were  randomly 
assigned  to  one  of  the  three  types  of  game  modifications. 

Procedure 

Jpproximately  half  of  the  subjects  in  each  of  the  two  eopertise  groips 
answered  the  quntiionnedre  and  took  part  in  two  different  bridge  studies 
(Experiments  2  and  3  of  this  report)  at  the  first  session  and  ocnpleted 
the  experimented  task  in  a  second  session;  for  the  other  half,  the  order 
was  reversed.  The  two  sessions  took  place  on  two  different  da^. 

Instructions  for  the  questionnaire  wexre  administered  orally. 
Presentation  of  the  questions  and  recording  of  the  answers  were  performed 
by  the  ocoputer. 

Instxu^ons  for  the  experimental  task  were  administered  via 
ocoputer.  Suln^ects  were  told  that  they  would  play  12  bridge  games  on  the 
ocoputer.  They  were  shown  how  to  enter  bids  and  plays  and  were 
familiarized  with  the  bidding  system,  which  was  simplified  Goren. 

Subjects  were  also  informed  that  there  was  no  carry-over  in  terms  of 
vulnerability  frcm  one  game  to  the  nexct.  They  were  told  that  they  would 
always  play  hand  South  and  that,  when  South  was  declarer  or  dummy,  the 
ocoputer  would  expect  them  also  to  play  hand  Nbrth. 

.Each  game  began  with  the  bidding.  When  declarer  and  dumoy  were 
determined,  the  player  to  the  left  (counterclockwise)  of  the  dumoy  played 
the  opening  lead.  Then  the  dumoy  cards  were  revealed.  When  each  trick  was 
ocnpleted,  the  cards  played  remained  on  the  "table"  for  further  analysis 
until  the  subject  hit  a  key.  The  player  who  won  the  previous  trick  led 
into  the  next  one  (exccept  in  the  lead-change  condition) .  When  the  subject 
had  finished  playing  all  13  tricks,  the  nexct  game  was  started. 
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Eeu::h  subject  received  the  senoe  12  games  in  different  randcndzed 
orders.  Games  were  blociked  with  regard  to  type  of  game  modificatlcn. 

Games  1  thrcuc^  3  and  10  through  12  were  regular,  undianged  games;  games 
4  throuf^  9  were  played  with  a  modification  of  the  familiar  bridge  rules 
(name  change,  rank'^nrder  change,  or  lea^i  change) .  Each  player  was  ea^xjsed 
to  only  one  form  of  rule  modifintian.  To  familiarize  thegoselves  wit^  the 
system,  subjects  played  two  practice  games  prior  to  the  e;q)erimental  task. 

VSien  they  had  ocnpleted  three  esqserimental  games,  players  were 
presented  with  the  new  rule  embedded  in  a  cover  story.  Ihe  cover  story 
was  identical  for  all  new  rules,  telling  subjects  to  imagine  tiienselves 
traveling  in  a  foreign  country  where,  one  evening,  they  were  invited  to  a 
card  game.  Predictably  enou^,  these  foreigners  played  a  game  that  was 
very  similar  to  bridge  with  only  elicit  modifications.  Appendix  C  gives 
the  ccnplete  cover  story  for  the  modification  of  card  and  suit  names.  In 
the  name-change  and  rank-order^-change  conditions,  the  eaqperimenter  changed 
the  k^  labels  on  the  ocnputer  keyboard  to  reflect  the  new  names  or  rank 
orders. 

Subjects  were  allowed  to  taka  as  much  time  as  they  needed  to  select  a 
bid  or  a  play.  Plays,  play  times,  the  number  of  tricks  won,  and  whether 
the  player  had  won  or  lost  the  game  were  recorded  by  the  ocnputer. 

Results 

Group  Assi^^nment 

l\iio  expertise  grocps  (experts  and  nonexperts)  were  formed  on  the  basis 
of  subjects'  assignments  to  the  espertise  catagoiy  system.  Subjects  were 
rated  fay  the  three  presidents  of  the  clubs  we  had  contacted.  Scores  for 
each  subset  were  obtained  by  averaging  across  raters'  assignments. 
Interrater  reliabilities  were  .82,  .85,  and  .89.  As  can  be  seen  in  Table 
12,  eagertlse  (esgxrts)  was  positively  oorzelated  with  self-reported  level 
of  skill  (Estlvl) ,  total  nunber  of  master  points  accumulated  (MPtot) , 
lumber  of  master  points  aocuimilated  during  the  past  two  years  (MPlast) , 
and  nunber  of  years  played  (Yr^yd) ,  but  not  with  age.  Subjects  with 
average  eoq)ertise  scores  of  1-4  were  assigned  to  the  ncnesqpert  group; 
those  with  scores  5  and  higher  formed  the  expert  group.  Ihe  particular 
cutoff  point  for  the  two  eopertise  categories  was  not  chosen  on  the  basis 
of  any  conceptual  meaning  but,  instead,  was  dictated  solely  by  the 
available  data  pool  (median  out) . 

Dichotomization  was  a  necessary  precondition  to  anedyze  data  by  \;ise  of 
anadysis  of  varlanoe.  (Siaple  regression  methods  would  not  have  required 
such  a  categorization  but  would  have  been  much  more  inconvenient  to  use. ) 
Table  13  provides  a  snapshot  of  eopert  and  nonexpert  characteristics.  As 
can  be  seen,  groips  differed  on  eoq»rtlse  scores,  on  self-estimated  level 
of  skill,  on  the  nunber  of  master  points  they  had  collected  in  total  and 
during  the  past  2  years,  and  on  the  number  of  years  they  had  played  bridge 
prior  to  toting.  Ihe  two  groups  did  not  differ  in  age. 


Insert  Tables  12  and  13  about  here 


Reliabilities  of  Dependent  Var< «)?!«? 

An  inportant  issue  that  we  wanted  to  address  prior  to  subsequent  data 
analysis  is  the  reliability  of  the  data.  Beca\]se  the  eoperimental  task  is 
a  rather  unusual  one,  the  obt2dned  data  might  have  lacked  internal 
oonslstency.  Because  the  specific  game  situations  to  which  subjects 
re^»nded  were  not  constant  across  subjects,  but  were  dependent  on  earlier 
plays,  ^lit*4ialf  reliabilities,  corrected  by  the  Spearman-Brewn  formula, 
were  oenputed  for  all  dependent  variables.  As  Table  14  shows,  the 
reliabilities  for  play  time  were  in  the  .80s  and  low  .90s,  which  is 
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generally  ocnsldered  to  be  hl^ily  satisfactory  for  data  analysis.  Data 
for  the  number  of  tricks  von  per  game  and  for  the  number  of  games  van 
generally  had  lower  reliabilities,  particularly  viien  unchanged  and 
modified  games  were  oonsideced  separately.  This  is  to  be  expected,  given 
the  categ^cal  nature  of  the  two  variables  and  the  differences  in 
strategy  that  subjects  of  different  skill  level  eooploy  to  deal  vdth  the 
rule  modificaticns. 


Insert  Table  14  about  here 

Oorrelationa  among  V;^Hab1ea 

A  second,  preliminary  issue  concerns  the  correlations  among  dependent 
variables.  Of  particular  interest  is  the  oorzelation  between  speed  and 
quality  of  subjects'  re^»nses.  Because  of  the  nature  of  this  stix^, 
error  rates  could  not  be  obtained.  Instead,  nunher  of  tricks  won  per  game 
and  number  of  games  won  were  \]sed  as  measures  of  subjects'  queditative 
task  performanoe.  As  can  be  seen  in  Table  15,  play  time  was  only 
moderately  correlated  with  the  two  qialltative  measures.  This  was  true 
for  both  imchanged  and  changed  games.  It  Should  be  noted  that  the  two 
overall  correlations  inclixted  all  observations  and,  thus,  did  not  suffer 
from  the  low  reliabilities  r^xrted  for  the  queditative  meeisures, 
especially  in  the  modified  games.  We  may  conclude,  therefore,  that 
streets  did  not  sacrifice  quality  for  speed  of  response. 

Ihe  overall  correlation  between  nunb^  of  trioks  won  per  game  and 
number  of  games  won  was  .36,  p  <  .001,  indicating  that  siA>jects  who  won 
more  tricks  also  vexe  more  likely  to  win  more  games  than  subjects  vho  won 
fcMer  tricks.  Semewhat  surprisingly,  however,  this  correlation  is  far 
from  perfect.  Winning  many  tricks,  it  appears,  does  not  necessarily  make 
for  a  good  player,  that  is,  a  playw  who  wins  many  games. 

Insert  Table  15  about  here 


Basic  Statistics 

Prior  to  inferential  data  analysis,  the  within-subjects  factor  of  game 
(1  to  12)  was  broken  down  into  four  blocks  of  three  games  each.  Ihus, 
blocks  1  and  4  ocntalned  only  games  that  were  played  laider  the  normed 
bridge  rules;  blocks  2  and  3  were  games  played  \aider  modified  rules. 

Tables  16  and  17  give  the  basic  statistics  for  edl  dependent  variables  in 
the  various  conditions. 

A  first,  informal  inflection  of  the  data  revealed  that  subjects'  play 
times  in  edl  efwrimental  conditions  were  faster  in  blook  4  than  in  any 
other  block.  Similarly,  subjects  won  fewer  tricks  and  fewer  games  in 
block  4  than  in  aiy  other  block.  Block  4  was  originally  designed  to  test 
whether  subjects  would  dif)lay  any  interference  effects  when  they  had  to 
return  to  the  original  bridge  rules.  Because  we  did  not  use  control 
groups  that  would  have  allowed  us  to  assess  and  partial  out  practice 
effects,  any  effect  of  retroactive  interference  could  have  only  been 
tested  if  block  4  performanoe  was  slower  than  block  1  perfarmnoe.  Ihis 
was  not  the  case,  however.  One  possible  reason  for  this  finding  micht 
have  been  subjects'  decreasing  attention  ^^an  at  the  end  of  the  task  due 
to  the  length  of  the  study.  (Average  toted  times  fent  on  the 
e3q»rimented  task  were  2  hours,  59  minutes  for  nonesq^rts  and  2  hours,  55 
minutes  for  eoqierts.)  Because  the  attention-deficit  typothesis  could  not 
be  ruled  out,  data  analysis  was  limited  to  subjects'  perfomanoes  in 
blocks  1  to  3. 
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Insert  Tables  16  and  17  about  here 


Base-line  Performance  fReoular  rules) 

Ihe  first  hypothesis  predicted  that  eKperts  vxxild  play  faster  and 
qualitatively  better  than  nonexperts  v^ien  regular  bricl^  rules  apply. 

Basic  statis^cs  for  es^ierts*  and  none3^)erts'  performances  in  the 
unchanged  games  (block  1)  are  shcMn  in  Table  18.  Because  the  dependent 
variables  were  occxelated,  a  multivariate  anzdysis  of  variance  was 
performed.  For  the  sain  effect  of  ejqsertise,  Wilks  L^.648,  F(l,28)=4.71, 
p<.01.  Ihe  effect  accounted  for  approxinately  35.3%  of  the  to^ 
variance.  Experts  played  faster  and  wen  more  games,  but  won  fewer  tricks 
per  game  than  did  noneiqperts.  To  decide  which  of  the  dependent  variables 
contributed  most  teward  the  discrimination  of  the  expertise  groups,  a 
follow-up  discriminant  aralysis  was  performed.  The  structure  coefficients 
of  the  discriminant  function  were  0.74,  0.41,  and  -0.51  for  the  dependent 
variables  of  play  time,  number  of  tricks  won,  and  number  of  games  won, 
respectively,  in^catii^  that  the  two  espertise  groups  differed  most  with 
regard  to  the  time  they  took  to  select  a  play. 


Insert  Table  18  about  here 


Effects  of  Rule  Modifications 

The  second  and  third  hypotheses  predicted  that  experts  would  generally 
be  more  eiffected  by  the  rule  modifications  than  nenesperts  would  be,  and 
that  they  would  be  more  ed!fected  by  the  deep-structural  rule  change  than 
by  the  two  surface-structural  changes.  We  espected  the  latter  pattern  to 
be  exactly  reversed  for  nQne)perts.  Thus,  nonesperts  should  be  more 
affected  by  the  surface-structural  changes  than  by  the  deep-struictural 
change. 

To  assess  changes  in  the  dependent  variables  that  were  due  to  the  rule 
modlflcsiticns,  percentage  scores  were  oemputed  by  dividing  each  subject's 
performance  in  the  changed  games  (averaged  across  blcxdcs  2  and  3)  by  his 
or  her  performance  in  the  regular  games  (block  1) .  We  oenputed  three 
percentage  scores  fesr  each  subject,  one  for  each  dependent  variable. 

Table  19  gives  the  mean  percentage  increase  or  decrease  for  the  dependent 
variables  in  adl  experimental  cxa^itlons.  A  multivariate  analysis  of 
variance  on  the  percentage  sc»res  revealed  a  significant  mean  effect  of 
eipertlse,  Wilks  1/^0.669,  F(l,28)=4.30,  p<.05,  accounting  for 
approximately  25.3%  of  the  total  variance.  A  contrast,  testing  the 
interaction  hypothesis,  was  also  significant,  WJJJcs  LsO.699,  F(l,28)=3.74, 
p<.05.  The  contrast  aoocunted  for  approximately  45.1%  of  the  total 
variance.  The  main  effect  of  rule  modification  and  the  (overall) 
interaction  of  expertise  with  rule  modification  were  both  nonsignificant, 

p>.10. 

locking  in  more  det2Lil  at  the  differential  effects  of  the  rule 
modifications  on  the  two  ejpertise  groups,  we  found  that  eoperts*  play 
times  increased  by  more  than  did  nonesperts*  as  the  new  rules  were 
introduced.  Also,  the  quality  of  experts'  play,  as  described  by  the 
number  of  games  won,  was  more  negatively  affected  than  was  nonesperts' . 

In  fact,  the  nonespeit  group  wan  more  games,  on  average,  vhen  playing 
under  the  new  rules  than  vhen  playing  under  the  original  rules.  However, 
the  nonexpert  group  wen  fewer  tricks  vhen  the  rule  changes  were 
introduced,  vhereas  the  experts  managed  to  increase  the  number  of  tricks 
they  won  d^ite  the  changes.  Increasing  familiarity  vdth  the 
experimental  setting  and  ^th  the  vray  the  oonputer  played  might  have  been 
reqscnsible  for  subjects'  gains  in  the  qualitative  measures  that  were 
observed,  despite  the  introduction  of  new  task  demands. 
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With  regard  to  the  significant  interaction  contrast,  we  found  that  the 
li^pothesized  pattern  of  results  was  obtained  for  each  of  the  three 
dependent  variables  individuadly.  As  Table  19  shows,  ea^ierts  were  more 
ed!fected  by  the  deep-structural  change  than  by  the  surface-structural 
changes  in  all  measures,  whereas  nonesqierts  were  more  affected  by  the 
surface-structural  chan^  than  by  the  deep-structural  change,  a  result 
that,  again,  showed  vp  in  all  dependent  meeisures. 

To  assess  which  of  the  dependent  variables  were  primarily  responsible 
for  the  sigxiif leant  results,  we  conducted  two  follow-^  discriminant 
analyses.  The  structure  coefficients  for  the  canonical  function  that 
separated  the  two  esqpertise  grexps  were  0.75,  0.55,  and  -0.25  for  the 
dependent  variables  of  play  time,  number  of  tricks  wen  per  game,  and 
number  of  games  won,  respectively,  indicating  that  the  differential  effect 
on  the  two  expertise  groups  showed  tp  most  cd.early  in  the  time  needed  to 
select  a  play. 

The  second  discriminant  analysis  (for  the  Interaction  contrast)  was 
done  separately  for  ejqperts  and  nonesperts.  For  eaqperts,  the  structure 
cxsefficients  obtained  were  0.97,  -0.23,  and  -0.16  far  play  time,  number  of 
tricdcs  won,  and  number  of  games  won,  respectively.  It  appears  that  the 
difference  between  surface  changes  and  deep-structural  change  is  primarily 
a  matter  of  time  for  ejperts.  In  other  words,  esperts  need  more  time  to 
deal  with  the  cieep-structural  change  than  to  deal  with  the  surface 
changes.  The  quality  of  their  play,  however,  does  not  seem  to  be 
differentially  ciffected  by  the  two  types  of  changes. 

For  noneoperts,  we  fexmd  structure  cx3effic:ients  of  -0.44,  0.21,  and 
0.66  far  the  dependent  measures  of  play  time,  number  of  trlclcs  won,  and 
number  of  games  won,  respectively.  For  less  eaperlenoed  bridge  players, 
therefcjre,  the  differenoe  between  the  t^pes  of  nile  changes  is  assexoiated 
with  a  differenoe  in  quality  of  play.  Nbnesperts  do  vx>rse  on  the  surface 
changes  primarily  with  regard  to  the  quality  of  their  performance. 


In  addition  to  assessing  the  overall  iapact  of  the  different  rule 
modifications  on  the  two  espertise  groups,  we  wanted  to  examine  how  well 
the  subjects  performed  the  reer  tasks  after  they  had  time  to  adapt  to  the 
changes.  One  way  of  addressing  this  question  is  to  oespare  subjects' 
performances  on  the  eaperimental  blocks  1  and  3  only.  We  therefore 
oanputed  new  percentage  scores  for  each  subject  by  taking  the  ratios  of 
block  3  performance  and  block  1  performance  on  all  dependent  measures.  A 
multivariate  analysis  of  variance  on  these  scores  revealed  no  significant 
effects,  indicating  that  both  expertise  groips  had  adapted  quite  well  to 
all  three  rule  changes.  In  fact,  esperts  appeared  to  have  von  back  their 
origined  ^)eed  and  quality  advantage  over  nonexperts  in  the  two 
surface-change  conditions.  Excluding  the  abstract-change  condition,  we 
obtained  a  marginal  main  effect  of  expertise,  Wilks  1^.791,  F(l,20)=2.29, 
pc.  10,  for  the  two  surface-change  conations.  Block  3  means  for  these 
conditions  are  shown  in  Table  20,  separately  for  the  two  expertise 
groips.  Wh^  oonpared  with  the  orlgined  performanoes  in  the  unchanged 
games  (conpare  Table  18) ,  it  appears  that  both  expertise  groips,  eifter 
only  a  few  practice  games  under  the  two  surface  r^es,  haii^  "recxrvered" 
already  in  terms  of  play  time  and  number  of  tricks  won.  The  only  variable 
on  vhich  both  grexps  did  not  reach  their  origincil  level  of  performance  was 
the  number  of  games  won. 


Insert  Table  20  about  here 
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Discussion 

E}^)eriinent  1  was  conducted  to  clarify  the  relation  between  ea^^ertise 
and  flffidbility.  We  had  hypothesized  that  two  variables,  namely,  the 
organization  of  the  knowledge  base  and  the  degree  of  prooeduraliration, 
would  affect  the  flexibility  of  a  problan  solver.  Ihe  main  results  of 
this  study  can  be  summarized  as  follows.  First,  changes  in  task  demands 
generally  affect  subjects  who  have  hi^tily  proceduralized  their 
problem-solving  strategies  more  than  those  \dio  rely  on  less  proceduralized 
strategies.  Esq^erts  are  generally  more  affected  by  the  rule  modifications 
than  noneoqperts  are.  Second,  changes  in  ta^  that  are 

incompatible  with  the  structure  of  the  knowledge  representation  have  more 
proncunoed  effects  on  subjects'  problem-solving  processes  than  have 
changes  that  are  ocnpatible.  Experts  were  more  eiffected  by  the 
deep-structural  change  than  by  the  two  surface  changes,  whereas  ncneaq^erts 
were  most  eiffected  by  the  suirfaoe  changes.  And  fin^y,  both  expertise 
groups  seem  to  be  able  to  adapt  their  existing  knowledge  bases  (^te 
rapi^y.  In  all  conditions,  with  the  exception  of  the 
deep-structural-rule-change  condition  for  esperts,  mibjects  reached 
approximately  their  original  performance  levels  aifter  only  a  few  games. 

It  is  interesting  to  note  that  the  difference  between  surface  and 
abstract  changes  for  expert  bridge  players  appears  to  be  most  pronounced 
in  the  play-time  variable,  vhereas  for  nonexpert  players,  the  difference 
between  these  two  ^pes  of  rule  modifications  seems  to  mamifest  itself 
most  clearly  in  a  quadity  measure,  namely,  the  number  of  games  won. 
^parently,  esperts  vise  the  ariditonal  time  they  need  when  dealing  with  the 
abstract  diange  to  enplcy  new  and  effective  game  strategies.  Thus,  their 
inflexibility  ml^t  reflect  a  difficulty  in  adapting  their  existing 
kncwledge  base,  rather  than  an  inability  to  Inhibit  older  solution 
l^ttems. 

Nonesperts,  on  the  other  hand,  seem  to  perform  particularly  poorly  in 
the  rank-order-diange  condition,  indicating  that  their  inflexibility  might 
be  primarily  due  to  their  inability  to  svppress  older  re^xsnses. 

Generally,  E}q)eriment  1  provides  moderate  svpport  for  the  hypotheses 
that  the  degree  of  prooeduralization  and  the  structure  of  the  knowledge 
base  eiffect  the  Inflexibility  of  a  cognitive  systan.  One  might  object 
that  it  is  not  sufficient  to  demonstrate  that  subject-grovps  that  organize 
their  kncwledge  bases  differently  and  rely  cn  different  degrees  of 
procedurallzation  also  differ  in  how  inflexible  they  are,  rather  that,  in 
order  to  claim  strong  svpport  for  our  hypotheses,  we  would  have  to  show 
that  subject-grovps  that  do  not  differ  on  the  relevant  variables  are  also 
equally  flexible  (or  inflexible) .  Study  2  was  designed  to  test  the  latter 
predic^on. 

E>periment  2 

In  the  second  ejq^eriment,  34  bridge  players  of  different  levels  of 
skill  were  instructed  to  generate  opening  bi^  to  given  bridge  hanis  as 
quickly  as  possible.  As  Qiamess  (1979)  pointed  out,  even  novice  bridge 
players  are  able  to  generate  a  reasonable  opening  bid.  Furthermore,  the 
choice  of  opening  bids  is  based  primarily  on  the  distribution  of 
honor-cards  and  on  the  number  of  tot£d  carcis  per  suit  in  a  given  hand. 
Althouf^  there  are  a  large  number  of  different  bidding  systems  to 
determine  a  bid,  these  systems  are  based  \pon  the  same  properties  of  hand« 
and,  generally,  do  not  arrive  at  different  opening  bids.  The  selection  of 
an  opening  bid  is  a  rather  cocplex  ccDputatlonal  process  that,  for  most 
players,  does  not  involve  ary  strategic  ocnsiderations.  With  regard  to 
op^iing  bids,  more  and  less  eaperlenoed  bridge  players  are  not  eopected  to 
differ  in  their  classification  of  hands  into  bid  categories.  Thus,  eopert 
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and  nonesqsert  bridge  players  do  not  use  differently  structured  kncwledge 
bases.  In  addition,  because  of  the  absence  of  strategic  oonsi derations , 
the  ocnputationed  process  of  selecting  an  opening  bid  can  be  esgiecbed  to 
be  prooeduralized  rather  quicikly. 

E>q)eriiDent  2  tested  three  hypotheses.  First,  es^erts  and  nanexperts 
will  select  identical  opening  bi^  in  the  same  amount  of  time  vhen  the 
original  bridge  niles  are  in  effect.  Second,  esqserts  will  generally  not 
be  more  affected  by  the  nde  changes  than  will  none>9)erts.  And  thixd, 
ejgierts  and  ncnesgierts  will  not  be  differentially  eiffected  by  the  rule 
modifications.  In  other  words,  there  will  be  no  significant  interaction 
between  esgsertise  and  l^pe  of  rule  change.  Because  the  selection  of 
opening  bi^  is  based  primarily  t^on  surface  features  of  hands,  we  expect 
both  esgjertise  grcn:^  to  be  more  affected  by  the  surface  changes  than  by 
the  deep-structureil  change. 

Method 

Subjects 

Subjects  were  the  same  as  in  E>g?eriment  1. 

Materials  and  Procedure 

Experimental  task.  For  the  bidding  task,  subjects  were  seated  in 
front  of  a  coDputer  screen  and  ke^xoard.  They  were  shown  slides  with 
"real”  bridge  hands  on  the  wall  just  above  the  ocnputer  screen  and  were 
Instructed  to  select  an  opening  bid  as  quickly  as  possible  by  pressing  one 
of  the  predesigned  keys  on  the  occputer  keyboard.  Bridge  hands  were 
di^layed  in  the  same  way  players  usually  hold  their  cards — with  the  cards 
fanned  out.  Subjects  were  told  that  th^  could  use  ary  bidding  system 
they  wanted,  but  that  they  should  use  it  consistently  during  the  whole 
session.  Slide  onset  and  offset  were  controlled  by  the  ccnputer.  As  soon 
as  subjects  had  re^xonded  to  a  slide,  the  next  slide  was  automatically 
di^layed  and  stayed  visible  until  the  opening  bid  was  selected. 

Subjects  responded  to  40  slides  (bridge  hands)  in  total.  Half  of 
these  slides  were  bid  under  normal  unchanged  bridge  rules;  the  other  20 
slides  were  bid  under  slightly  modified  bridge  rules. 

Types  of  rule  modifications,  lypes  of  rule  modifications  were  the 
same  as  those  used  in  Experiment  1.  Ihey  were:  a  name  change,  a 
rank-order  change,  and  a  lead  change.  In  the  name-change  condition,  the 
familiar  card  and  suit  names  (ace,  king,  queen,  jack,  and  spade,  heeut, 
diamond,  club)  were  replaced  ^  ncnfamili^,  nonsense  names  (nitz,  lork, 
dill,  beib  for  cards,  and  pular,  biref,  ramog,  kamer  for  suits, 
respectively) .  Ihe  rank-order  change  modified  the  familiar  card  and  suit 
rank  orders  (ace,  king,  queen,  jack  to  king,  jack,  ace,  queen, 
respectively,  for  cards,  and  ^>ade,  heart,  diamond,  clilb  to  diamond, 
heart,  club,  spade,  respectively,  for  suits.)  Ihe  lead  change  modified 
vho  began  each  play.  Instead  of  the  player  who  won  the  last  trick,  the 
player  with  the  lowest  card  in  the  last  trick  led  into  the  next  trick. 

Slides  were  blocked  with  regard  to  the  rule  modifications.  The  first 
10  and  the  last  10  slides  were  bid  under  normal  conditions,  the  rest  voider 
one  of  the  modified  oonditions.  Each  subject  received  only  one 
modification  in  the  task.  Bids  and  reaction  times  were  recorded  by  the 
ccnputer. 

D^SD 

Independent  between-subjects  variables  in  this  task  were  level  of 
eppertise  (nonexpert,  eppert)  and  ^pe  of  rule  modification  (name  change, 
rank-order  change,  lead  change) .  Slide  (1  to  40)  was  a  within-subjects 
variable.  In  each  epqpertise  group,  subjects  were  randcmly  assigned  to  one 
of  the  chpree  types  of  game  mo^fications,  with  the  restriction  that  no 
subject  received  the  same  type  of  change  she  had  alreacfy  experienoed  in 
Epperiment  1. 
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As  Table  21  shows,  all  of  the  Item  and  subject  leliabllitles 
(Cranbach's  alpha)  for  the  dependent  variable  bid  tixe  were  in  the  hi^ 
.80's  and  .90's,  which  is  considered  very  satisfactory  for  data-analytic 
purposes. 


Insert  Table  21  about  here 


Basic  statistics  for  bid  tines  in  the  various  eMperinental  ocnditions 
are  shown  in  Tedale  22.  Means  are  listed  aa  a  function  of  es^ertlse,  ^pe 
of  rule  nodiflcaticn,  and  block.  As  in  the  previous  experinent,  the 
within-subjects  factor  of  slide  (1  to  40)  was  broken  down  into  four  blocks 
of  10  slides  each.  Blocks  1  and  4  contained  only  slides  that  were  bid 
under  the  nonaal  bridge  rules;  blocks  2  and  3  oontadned  slides  bid  imder 
modified  rules.  Hypothesis  1  predicted  that  all  subjects  would  choose 
identical  opening  bids  in  the  sane  amount  of  time.  A  oonparison  of 
experts'  and  nQne;g)erts'  bid  times  prior  to  any  rule  modification  (block 
1)  revealed  vhat  we  had  expected.  Although  eoqperts  bid  faster  than 
nonej^ierts  (7.79  sec.  for  ej^^rts,  9.51  sec.  for  nonea^erts, 
respe^vely) ,  this  difference  was  not  statistically  reliable, 
F(l,28)*2.73,  p>.10.  J^^parantly,  all  subjects  had  prooeduralized  the 
strata^  to  select  an  opening  bid  to  rou^y  the  same  extent. 

Intern-subject  agreement  with  regard  to  which  bid  to  choose  was  very  hi^, 
90.10%  and  79.22%  for  the  noneagert  and  expert  groi^,  respectively.  In 
addition,  the  opening  bid  most  favored  by  the  ncneogserts  was  identical  to 
the  one  preferred  by  the  eogierts  for  all  twenty  brii^  hands  that  were  bid 
under  the  orlginad  niles.  Thus,  eoqperts  and  noneoperts  tended  to  select 
identical  opening  bids  in  approximately  the  same  amount  of  time. 


Hypotheses  2  and  3  predicted  'Oiat  experts  and  none^qerts  would 
generally  be  equally  affected  by  the  rule  modifications  and  that  both 
grcxps  would  be  more  eif  fected  by  the  surface  modlflcatlcns  tiian  by  the 
abstract  change.  As  in  Experimgit  1,  where  we  assessed  changes  in  the 
depCTident  variables  that  were  due  to  the  rule  modifications,  we  conputed 
percentage  scores  by  dividing  each  subject's  bid  time  in  the  changed  games 
(averaged  across  blocks  2  and  3)  by  her  bid  time  in  the  regular  games 
(averaged  across  blocks  1  and  4) .  An  analysis  of  variance  on  these  scores 
reveeded  only  a  significant  effect  of  type  of  nile  modification, 
F(2,28)«3.59,  p<.05,  accounting  for  19.64%  of  the  total  variance.  Figures 
3  and  4  show  that  edl  subjects  were  more  affected  by  the  surface  changes 
than  by  the  deep-structural  change.  Also  in  accordance  with  the 
hypotheses,  the  main  effect  of  eoq»rtise  and  the  interaction  of  expertise 
with  type  of  rule  modification  were  not  significant,  p>.10. 

Insert  Figures  3,  4  about  here 

PlsstsgAth 

Ihe  major  purpose  of  E:q»riment  2  vas  to  sipport  our  claim  that 
eaq^ertise  and  flesdbility  are  strongly  linked  in  a  cognitive  system. 
Experiment  1  had  demonstrated  that  different  degrees  of  prooedurallzatlon 
and  different  kncwledge-^sase  structures  lead  to  differences  in  the  ease 
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with  which  subjects  can  deal  with  novel  task  demands,  m  Experiment  2,  we 
shewed  that  subject-grexps  that  do  not  differ  cn  these  two  variables  also 
do  not  differ  in  hew  flexibly  they  can  use  their  existing  knowledge.  Ihe 
results  of  Experiment  2  strengthen  our  cledm  that  the  degree  of 
prooeduralizaticn  and  the  strrtcture  of  a  knowledge  base  eiffect  the 
flexibility  of  a  cognitive  system. 

Ejgseriment  3 

In  Experiments  1  and  2,  we  were  primarily  ocnoemed  with  the 
differential  impact  of  surf  ace-structural  and  deep -structural  rule 
modif icatiens  cn  espert  and  ncne}p«±  bridge  players  and  with  the 
mechanisms  of  interference  in  the  lead-change  ocnditicn.  So  far,  hewever, 
we  did  not  discuss  the  mechanisms  of  interference  for  the  two 
surfaoe-stnictural  rule  modif  icatiens.  Khereas  subjects'  problons  with 
the  lead  change  can  only  be  eapledned  in  terms  of  ocnoeptual  levels  of 
processing,  subjects'  difficulties  with  the  name  and  rank-order  changes 
mi^xt,  either  in  addition  to  interference  cn  the  ocnoeptual  level  or 
solely,  be  attributed  to  perceptual  phenanena.  It  sd^t  be  argued,  for 
instance,  that  the  surface  modifications  affected  all  subjects  primarily 
at  the  level  of  encoding  and  only  to  a  minor  degree  at  a  conceptual 
level.  Or  alternatively,  that  experts  were  most  effected  on  a  perceptual 
level  whereas  nonesperts  were  most  eiffected  on  a  conceptual  level  or  vice 
versa.  E>periment  3  was  designed  to  test  the  effects  of  the  name-change 
and  rank-order-change  conditions  at  the  level  of  encoding. 

In  Experiment  3,  players  of  differing  skill  levels  were  shown  slides 
of  bridge  hands  (13  cards  displayed  in  a  fanned  position)  in  exactly  the 
same  way  they  usu2dly  perceive  bridge  hands.  Slides  were  visible  for  only 
5  seconds.  After  slidi^ffset  players  were  asked  to  vnrite  down  as  many  of 
the  cards  as  they  could  remember.  They  had  the  option  of  watching  the 
same  slide  as  often  as  they  wanted  to  until  they  had  written  down  all  13 
cards  of  the  seen  hand.  Half  of  the  bridge  hands  shown  were  structured  in 
the  same  way  players  usually  strricture  their  hands;  for  the  other  heQf, 
features  of  the  visu2d  dl^xlay  were  changed  so  as  to  oorre^xond  to  the 
name-change  ard  rank-order-change  conditions  used  in  Experiments  1  and  2. 
In  the  name-change  condition,  the  letters  on  honor-cards  were  ch2tnged  to 
new  ones.  In  the  rank-order  condition,  the  familiar  rank  orders  of  suits 
and  hl^  cards  were  changed;  conseguently  hand  di^lays  mirrored  the  new 
rank  orders.  Reaction  time,  number  of  cards  written  d^m  per  trial,  and 
number  of  cards  correctly  identified  per  tried  were  recorded. 

Method 

Subjects 

Subjects  were  the  same  as  in  Experiments  1  and  2. 

Materials  and  Procedure 

Exoaerimental  task.  Subjects  were  seated  in  front  of  a  computer 
screen  and  keyboard.  Ihey  were  shown  slides  with  bridge  hands  (13  cards 
per  hand)  on  the  wall  just  above  the  oceputer  screen  in  a  distance  of 
apprcodmately  1.50  meters.  They  were  told  that  the  teidge  hands  would 
stay  visible  for  5  seconds  and  that  they  should  try  to  memorize  as  maixy 
cards  as  they  could.  Slide  onset  and  offset  were  controlled  by  the 
computer.  As  soon  as  a  slide  disappeared  subjects  were  prompted  to  write 
down  as  many  cards  as  tiiey  could  remember  vising  special  predesigned 
on  the  computer  keyboard.  Subjects  had  the  option  to  view  a  sll^  as  often 
as  they  wanted.  They  wexce  instrvicted  to  watch  a  slide  as  often  as  they 
needed  to  write  down  all  cards  of  the  bridge  hand. 


CNR  Ocntract  N0001485K0589  Final  Report,  Page  30 


Subjects  respcnded  to  40  different  slides  or  bridge  hands.  Half  of 
these  slides  were  shown  adapting  the  ocDmcnly  vised  hand  structuring 
(ordered  for  suits  and  rank  of  card  within  ^t) ;  for  the  other  20  slides 
the  visual  display  was  changed  in  one  of  two  ways. 

Types  of  display  modifications.  Ihe  two  types  of  rule  modifications 
used  in  Eagariment  3  were  the  two  surface  changes  vised  in  Experiments  2 
and  3.  Th^  were  a  name  change  and  a  rank-order  change.  In  the 
name-change  cxndition,  the  famtl  1  ar  card  and  ajit  names  (aoe,  king,  queen, 
jack,  and  spacie,  heart,  diamond,  club)  were  replaced  1:^  nonfamiliar, 
nonsense  names  (rutz,  lark,  dill,  beib  for  cards,  and  pular,  biref,  ramog, 
kamer  fear  suits,  respectively) .  Ihus,  the  letters  appearing  in  the 
comers  of  a  card  identifying  acae  (A) ,  king  (K) ,  (jueen  (Q) ,  and  jack  (J) 
were  replacad  by  R,  L,  0,  and  B  reflectively,  m  the  rank-order-change 
cxnditlon,  the  familiar  card  order  (ace,  king,  ejueen,  jack)  was  changed 
to  king,  jack,  ace,  queen,  and  tiie  familiar  suit  rank  order  (^ac3e,  heart, 
diamond,  cd.vib)  was  changed  to  ciiamend,  heart,  clvib,  spede.  In  this 
cxndition,  only  the  structuring  of  cards  within  suits  was  changed  in  the 
displeq^.  The  third  canditicn  was  a  control  cxndition  in  which  subjects 
saw  only  regularly  strvictured  hands. 

P^lgn 

Dqjendent  variables  were  the  nunber  of  cards  subjects  wrote  ciown 
aifter  each  5-sec.  glance  (Total) ,  the  nunber  of  cards  they  icientified 
correctly  (Oorxect) ,  and  the  average  time  they  needed  to  write  down  the 
cards  th^  remembered  (RT) .  Indepfxient  betueen-subjects  variables  were 
level  of  expertise  (nonexpert,  expert)  and  type  of  display  change  (name 
change,  rank-order  change,  cxstrol) .  Slide  (1  to  40)  was  a  within-fibjects 
variable.  In  each  espertise  group,  svbjects  were  randcnly  assigned  to  one 
of  the  three  types  of  modifications  with  the  restriction  that  no  subject 
received  the  type  of  modification  she  had  ejqperienced  in  Ejqperiments  1  and 
2. 

Results 

Because  some  of  the  subjects  were  able  to  identify  all  13  cards  of  a 
bridge  hand  after  their  first  5-sec.  glance,  data  anedysis  was  limited  to 
subjects'  first  trial  on  each  slide. 

_ _  _ _  «■ _ ^  aa _ J  _  a  .  _ 


Table  23  lists  the  reliabilities  for  all  dependent  variables  on  beth 
ncrmal-dl^ilay  and  modifled-dleplay  trials.  As  can  be  seen,  most  of  the 
reliabilities  are  in  the  hl^  70 's  or  above.  Reliabilities  for  the 
modlf ied-di^lay  trials  are  generally  lower  than  thesse  for  the 
normal-di^lay  trials.  This  is  particadarly  true  for  the  subject 
reliabilities.  Overall,  hewever,  the  reliabilities  are  quite  satisfying. 

Insert  Table  23  about  here 


Table  24  shews  the  (xrzelations  between  all  pairs  of  dependent 
variables.  Of  particular  Interest  is  the  correlation  between  speed  and 
(quality  of  subjects'  re^ionses.  As  can  be  seen,  reaction  time  (RT) 
caorrelated  significantly  negatively  with  the  nunber  of  cards  subjects 
wrote  dewn  (Total)  and  with  the  mmher  of  cards  they  remenbered  correctly 
(Correct) ,  indicating  a  Epeed-acxuracy  tradeoff  betwen  speed  and  equality 
of  subjects'  responses.  The  faster  mibjects  responded  the  more  likely 
they  were  to  falWly  identify  cards.  The  significant  carrelations  among 
the  ciependent  variables  point  to  the  use  of  multivariate  techniepes  as  the 
appropriate  means  of  data  analysis. 
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Insert  Table  24  about  here 


Basic  stati stirs 

As  in  Experiments  1  and  2,  the  withinr*subjects  factor  of  slide  (1  to 
40)  vfas  broiken  dcMi  into  four  blocks  of  20  slides  each.  Basic  statistics 
for  e9$)erts'  and  novices'  perfomanoes  in  the  four  blocks  are  shown  in 
Tables  25  and  26.  Means  on  each  of  the  three  dependent  variables  are 
listed  as  a  function  of  t^pe  of  display  change  and  block. 


Insert  Tables  25  and  26  about  here 


Base-line  Performance 

Table  27  shews  the  means  for  the  two  eaq^ertise  groips  on  each  of  the 
three  dependent  variables  for  those  trials  that  were  displayed  under 
regular  conditions.  Because  the  dependent  variables  were  significantly 
correlated,  a  rultivariate  anedysis  of  variance  was  performed.  For  the 
mean  effect  of  esxpertlsA,  Wills  L><}.764,  F(l,32)»3.09,  p<.05.  Ejqperts 
re^xnded  faster,  wrote  down  more  cards  in  toted,  and  correctly  identified 
more  cards  than  did  novices.  m  order  to  decide  which  of  the  dependent 
variables  contributed  most  toward  the  discrimination  of  the  ejqsertise 
groeps,  a  follcw^tp  discrizdnant  analysis  was  conducted.  Because  there 
were  only  two  groups  to  separate,  only  one  non-zero  eigenvaliie  could  be 
obtained,  accounting  for  epproximately  8%  of  the  total  variance.  The 
stnxoture  coefficients  of  the  discriminant  function  were  -0.38,  0.83,  and 
0.93  for  the  dependent  variables  of  RT,  Total,  and  Oorrect,  respectively, 
indicating  that  experts'  encoding  superiority  reveals  Itself  more  clearly 
in  the  amount  of  information  they  can  take  in  than  in  the  time  they  need 
to  do  so. 


Insert  Table  27  eOxut  here 


Changes  in  Encodincr  Ability  due  to  Modifications  of  the  visual  Display 

Table  28  depicts  the  mean  percent  increases  or  decreases  in  the  three 
d^)endent  variables  (relative  to  individual  base-line  performaix^s)  for 
the  two  expertise  groups  in  all  experimental  conditions.  A  two-way 
multivariate  analysis  of  variance  with  expertise  and  t^pe  of  display 
modification  as  independent  variables  found  only  the  main  effect  of  type 
of  display  change  to  be  significant,  Wilks  L=0.58,  F(2,28)«2.72,  p<.05). 
Both  display-change  conditions  differed  fron  the  control  condition  (Wilks 
L»0.74,  F(l,  28)»3.08,  p<.05  for  the  name-change  oonditlcn;  Wilks  Dso.76, 
F(l,28)=2.73,  p<.07  fca:  the  rank-order-change  condition),  but  not  frem 
each  other.  Ihe  main  effect  of  espertise  and  the  intereuction  of 
expertise-type  of  di^lay  change  were  not  significant.  Thus,  experts  and 
nonexperts  did  not  differ  in  the  exetent  to  which  they  were  eif fected  by  the 
di^lay  changes  in  any  of  the  experimented  conditions. 


Insert  Table  28  about  here 


Discussion 

The  three  major  iresults  of  Experiment  3  can  be  summarized  as 
follows:  First,  replicating  Chamess's  (1979)  earlier  findings  for  the 
domain  of  bridge,  we  found  that  experts  encoded  meaningful  Information 
faster  and  with  fewer  errors  than  did  nonexperts.  Second,  the  two  types 
of  display  changes  did  not  differ  in  their  effects  on  subjects'  encoding 
abilities.  And  finally,  experts  and  nonexperts  were  not  differentially 
affected  by  the  two  typ^  of  changes  of  the  visual  di^lay. 
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Ihe  fact  that  experts  and  nonexperts  were  not  differentially  zdffected 
by  the  two  types  of  changes  has  iapcxtant  implications  for  the  discussion 
of  Experiments  1  and  2,  suggesting  that  the  effects  of  the 
surface-structural  rule  modifications  obtained  in  these  studies  mic^,  for 
both  espertise  grcxps,  be  partly  due  to  the  disruption  of  normally 
oocuring  encoding  processes.  Ihe  findings  of  Eaperiment  3  do  not  sipport 
the  argument  that  the  surface  changes  affected  esqperts  primarily  on  a 
perceptual  level  and  nonesperts  primarily  on  a  conceptual  level  of 
information  processing  or  vice  versa. 

General  Discussion 

The  three  esperlments  described  in  this  report  were  conducted  to 
ejplore  the  relation  between  expertise  and  flea^lllty.  Studies 
imrestigating  the  nature  and  develop^nt  of  expertise  typicedly  focused 
upon  the  benefits  of  mastery  in  a  domain,  that  is,  they  e>ddressed  the 
question  of  why  it  is  that  esperts  perform  more  q^cskly  and  better  than 
nonesperts  do.  In  contrast,  we  argued  that  greater  eapertise  does  not 
necessarily  result  in  superior  performance  in  all  instances.  In  fact,  we 
hypothesized  that  the  very  same  mechanisms  that  are  respcnsible  for 
e}perts'  sometimes  amazing  performanoes  on  conventional,  familiar  tasks 
might  work  against  them  when  they  hacve  to  deal  with  novel  tai^  demands. 
Specifically,  we  assumed  that  two  variables,  the  degree  of 
procedurali^tion  and  the  organization  of  the  knowledge  base,  can  cause 
two  different  types  of  inflei^llity,  the  inablli^  to  inhibit  existing 
response  patterns,  and  the  Inability  to  modify  an  existing  system  of 
knowledge. 

The  results  of  the  research  presaited  in  this  report  can  be 
summarized  as  follows.  First,  problem  solvers  vho  have  procaduralized 
their  solution  strategies  are  less  flexible  than  ones  who  have  not. 

E}perts  were  generally  more  eif fected  than  hcnesperts  vhen  task  demands 
changed.  Second,  problem  solvers  are  less  flea^le  when  dealing  with  task 
modifications  that  are  incxmpatible  with  the  structure  of  their  knowledge 
representations  than  when  dealing  with  changes  that  are  cxnpatible.  W<e 
found  that  esperts  were  more  affected  deep-structural  changes,  \A)ereas 
noneiperts  were  more  eiffecPed  by  surface  changes  when  the  two  groups 
stmctured  their  knowledge  representaticns  differently.  Vlhen  both  groups 
operated  on  the  same  knowledge  base,  they  were  not  differentially 
affected.  Third,  euLl  subjects  were  able  to  adjvist  to  new  task  demands. 

In  most  oondlticms,  they  returned  to  their  crigined  levels  of  performance 
quite  quickly.  And  finally,  the  effects  of  the  two  surface-structured 
rule  manipulations  mi^t  for  all  subjects  be  partly  due  to  perceptual, 
rather  than  cxno^tual  phencmena. 

On  a  general  level,  it  can  be  argued  that  the  findings  reported  in 
this  report  are  due  to  negative  interference  of  previously  acxjuired 
knowledge  on  the  acxjuisiticn  of  new  knowledge.  Ihe  basic  tarm  of  this 
interference  mi^it  be  cxsnsidered  a  variatlcn  of  the  ocmmon  A-B,  A-C 
framework,  used  in  studies  of  associative  learning.  Bower  (1974)  pointed 
out  that  interference  theory  need  not  be  restricted  to  meaningless 
material  or  to  "factual  details."  In  our  work,  subjects  esperienoed 
interference  vhen  they  were  required  to  reason  with  partly  novel  oonoepts, 
with  information,  that  is,  that  was  only  partially  meaningful  in  terms  of 
prior  kncwledge. 

nils  particular  interpretation  is  very  closely  related  to  Sternberg's 
(1981,  1982)  discussions  of  "nonentrenched"  oonoepts  and  Tetews]^  and 
Sternberg's  (1986)  work  on  the  conc^kual  and  lexical  determinants  of 
nonentrenched  thinking.  Theoretical  accounts  of  negative  interferenoe 
eflects  in  associative  learning,  as  mentioned  in  the  introcbcticn,  have 
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enphaslzed  different  ccnponents  of  flexibility.  Proactive  interference 
(e.g. ,  Underwood,  1957)  and  list  dicrimination  theories  (e.g. ,  Sternberg  & 
Bower,  1974)  describe  negative  interference  effects  primarily  in  terms  of 
subjects'  inability  to  inhibit  older,  previously  established  associations, 
whereas  organizationed.  theory  (e.g. ,  Utlving,  1966)  tries  to  link  than  to 
adaptation  difficulties.  Our  data  suggest  that  the  ^pe  of  interference 
that  occurs  mi^t  be  related  to  the  organization  of  prior  knowledge  and  to 
the  extent  that  proceduralized  strategies  are  being  used,  ^parently, 
problem  solvers  with  less  structured  and  less  proceduralized  knowledge 
systems  (naneaqperts)  have  more  difficulties  ij^iibiting  old  responses  than 
they  have  modifying  their  knowledge  r^uresentations.  Problem  solvers  vbo 
rely  on  hic^y  structured  and  proceduralized  knowledge  systems  (eiperts) 
on  the  other  hand,  seem  to  experience  most  difficulty  vben  modifying  prior 
knowledge. 

The  experimental  findings  reported  hers  can  also  be  discussed  with 
iregard  to  recent  theoretical  developments  in  the  field  of  intelligence. 

It  has  long  been  a  traditional  belief  that  intelligence  can  be 
characterized,  in  part,  as  the  ability  to  acquire  new  information. 

Seveiral  theorists  have  proposed  that  the  ability  to  deed,  with  novel,  and 
particularly,  partiailly  novel  information  is  an  integrad.  aspect  of 
intelligence  (e.g.,  Berg  &  Sternberg  1985;  Cattell,  1971;  Horn,  1968; 
Raaheim,  1974;  Snow,  1981;  Sternberg,  1985).  The  present  findings  suggest 
that  this  ability  can  reflect  inter-kibject  differences  in  domadn-^jecific 
strategies  and  categorizaticxx  of  knowledge. 

This  Interpretation  also  lends  itself  very  easily  to  a  phencmenon 
that  has  been  r^xxrted  fireguently  in  the  developmental  literature:  the 
decline  of  fluid  intelligence  with  age  (Horn,  1968) .  Our  data  suggest 
that  this  decline  mi^it  be  due,  at  least  in  part,  to  the  proceduralization 
of  responses  to  highly  recognizable  and  familiar  situations,  and  to  the 
organization  of  existing  knowledge  bases.  In  other  words,  by  being  more 
"expert"  at  performing  a  range  of  familiar  tasks,  older  indivldueds  mi^t 
experience  an  irpairment  in  performance  vhen  a  novel  task  conpetes  with 
vhat  is  already  known. 

In  summary,  the  research  presented  hexre  touches  upon  several 
important  issues  within  cognitive  psychology.  We  hope  to  have 
demonstrated  that  the  costs  of  expertise  can  be  fimitfuUy  exploited  not 
only  to  unravel  the  nysteries  of  mastery  tut  ed.so  to  xaiderstand  basic 
cognitive  principles. 

Conceptual  and  Lexciced  Determinants  of  Nonentrenched  Thinking^ 

We  irely  on  cur  knowledge  of  the  world  to  make  inferences  and  to  form 
expectations.  The  various  schemata  and  conc^xts  that  eae  stored  in  our 
memories  permit  us  to  understand  the  ccaoplex  relationships  that  we 
exqxerlenoe  in  our  everyday  lives.  Althou^  our  knowledge  edlcws  us  to 
deed  with  a  broad  range  of  situations,  there  are  limitations  to  its 
usefulness — vhen  we  encounter  something  that  is  unfamiliar,  such  that  it 
fails  to  conform  to  our  expectati(ns,  we  tend  to  make  inooiTect 
infeirences.  These  errors  reflect  a  fundamental  aspect  of  human  thcu^t 
because  vhenever  we  exqxerierxce  something  new,  we  have  to  deal  with  it  in 
terms  of  vhat  is  familiar  and  old-fai^oned  (Oppehheimer,  1956) . 

The  question  of  vhy  we  find  it  difficult  to  make  vadid  inferences  in 
unfamiliar  situations  is  implicit  in  Goodman's  (1955)  "new  riddle  of 
induction."  Goxodman's  {hiloxsophical  argument,  reviewed  by  Styrms  (1975) 
and  by  Sternberg  (1982) ,  implies  that  oxertain  events  are  quite  natural  and 
others  are  quite  unnatural.  We  expect  the  former  kinds  of  events,  but  not 
the  latter  kinds  of  events,  to  oxcoxur.  Goxoxdman  argued  that  as  a  result  of 
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cur  cultuzel  esq^erienoes,  we  develop  certain  terns  to  deal  with  our 
everyday  lives.  These  "entrenched  predicates"  cause  us  to  favor  oertedn 
inferences  over  others  even  thcu^  the  acvzdlable  evidence  scpports  several 
different  oondvisions. 

Goodman's  philosophical  ideas  about  "entrenchment"  are  related  to  a 
psychological  phencnenon  that  was  first  eaqilored  by  Barlett  (1932) .  The 
subsets  in  Bartlett's  e^qeriment  were  recjrired  to  remember  stories  that 
came  fren  a  foreign  cultural  and  social  environment.  The  nature  of  these 
stories  caused  su^ects  to  make  a  number  of  systematic  errors.  Bartlett's 
work  has  traditioneOly  been  used  to  stpport  the  idea  that  when  we  recall  a 
teset,  we  remember  a  general  schema  and  we  use  this  schema  to  reconstruct 
the  specific  details  of  the  text.  However,  his  work  edso  suggests  that 
once  we  have  a  knowledge  system  'ttiat  describes  how  the  world  is  ordered, 
it  is  often  quite  difficult  to  consider  8d.temative  ways  of  thinking. 

This  interpretation  is  compatible  with  the  work  of  Mai^  (1931)  and  of 
Duncker  (1945) ,  vbo  showed  that  prior  knowledge  can  impede  problem  solving 
throu^  the  medianism  of  "functional  fixedness." 

Goodman's  philosophical  ideas  about  entrenched  predicates  appear  to 
have  soB»  interesting  psychological  implications.  The  studies  cited  above 
suggest  that  when  we  try  to  co^irdiend  information  that  is 
"nonentrenched, "  we  experience  interference  from  our  prior  knowledge. 
HcMsver,  it  is  important  to  specify  the  precise  meaning  of  nonentrenchinent 
and  the  eiact  nature  of  the  associated  interference.  In  particular,  it  is 
necessary  to  determine  whether  nonentrenchment  is  different  from 
unfamlliarlty,  and  it  is  also  necessary  to  specify  the  conditions  under 
vhich  prior  knowledge  will  cause  interferenoe  in  processing  nonentrenched 
ocncepts.  The  research  r^xsrted  in  this  a3±icle  was  designed  to  address 
these  issues  by  6}q)loring  the  psychologlced  reality  of  ncnentrenchment. 

Several  attempts  have  been  made  to  distinguish  between  "natural"  and 
"vinnatural"  concepts  and  between  entrenched  and  nonentrenched  concepts. 
Rips  (1975)  dealt  with  the  nature  of  entrenched  concepts  by  showing  that 
representative  (or  prototypical)  instances  of  a  category  facilitate 
inductive  generalizations,  compared  to  less  representative  (or 
prototyplcad.)  instances  of  a  category.  Hs  therefore  equated  entrenchment 
with  representativeness.  Osherson  (1978)  formally  oonsidered  the  various 
theoretical  Issues  that  ml^bt  be  de^t  with  in  distinguishing  between 
natural  and  unnatural  oono^jts.  For  exanple,  he  suggested  that  natural 
oonoepts  are  more  likely  to  figure  in  law-like  generalizations  (i.e.,  they 
are  mere  "projectible")  and  that  they  should  fai^itate  deductive 
reasoning  in  oemparison  to  immatural  concepts.  Oshersen's  work  was  mainly 
theoretical,  rather  than  empirical,  emphasizing  the  formal  constraints  on 
natural  oonoepts.  Finally,  Keil  (1979)  has  argued  that  natural  oonoepts 
are  formed  out  of  basic  "ontological  categories,"  which  refer  to  the  most 
general  categories  of  things  in  the  world.  For  exanple,  a  natural  oonoept 
M^t  oontadn  the  category  of  all  living  things,  whereas  an  unnatural 
concept  ml^txt  be  made  of  humans  and  plants,  without  also  Including 
nonhuman  animals.  Kell's  work  therefore  aiphasizes  the  stnactural 
differences  between  natural  and  unnatural  oonoepts. 

In  contrast  to  these  analyses,  Sternberg  (1982)  directly  studied  the 
processing  implications  of  natural  and  unnatural  conceits  by  designing  a 
reasoning  task  based  on  Goodman's  (1955)  new  riddle  of  Indu^on.  For 
exanple,  in  one  eoperiment,  an  unnatural  oonoept  was  described  by  the  word 
"grue,"  which  referred  to  an  object  that  was  green  in  the  year  1977  and 
turned  blue  in  the  year  2000.  The  corresponding  natural  oonoept,  vbich 
did  not  involve  a  transformation,  was  described  by  the  term  "green."  This 
term  referred  to  an  object  that  was  ptyslcally  green  in  the  year  1977  and 
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Mas  also  green  in  the  year  2000.  Sternberg  proposed  three  alternative 
hypotheses  to  ei^edn  uhy  natural  cenoqyts^  such  as  green,  be  easier 

to  process  than  unnatural  ones,  such  as  grue:  (a)  natural  concepts  belong 
to  the  conceptual  systean  with  vhich  we  grew  up  and  so  they  are  nore 
entrenched  in  cur  ea^ierienoe;  (b)  natural  conceits  do  not  carry  an 
esgiectation  of  change  and  so  they  are  "simpler,"  psychologically,  to 
process;  and  (c)  the  effects  of  sinplici^  and  entrenchment  may  operate 
jointly  such  that  natural  concepts  may  be  both  simpler  and  more  entrenched 
in  our  eaperienoe.  The  results  of  Sterhberg's  research  indicated  that 
both  entrenchment  and  simplicity  contributed  to  ease  of  infarmation 
processing,  but  that  the  effect  of  simplicity  was  greater  than  the  effect 
of  entrenchment. 

The  present  work  was  imidertaken  to  extend  Sterhberg's  (1982)  work  by 
clarifying  the  nature  of  nonentrenchment.  A  reasoning  was  designed  in 
vhich  the  naturalness  of  a  concept  and  the  ^pe  of  name  used  to  define 
that  canoept  could  be  ind^iendently  varied.  A  conceptual  system  was  found 
in  vhich  the  content  could  be  expressed  in  four  different  forms,  such  that 
two  levels  of  concepts  (natural  or  immatured)  could  be  crossed  with  two 
levels  of  names  (familiar  or  novel) .  The  underlying  assumption  for  this 
design  was  that  concept  naturalness  and  lexic2d  familiarity  mi^xt  be 
important,  singly  or  in  ocinbinatim,  in  distinguishing  between  entrenched 
and  nonenbrenched  oonoq±s. 

In  the  first  esperimient  to  be  described,  subjects  were  required  to 
solve  reasoning  problems  in  vhich  they  had  to  select  among  alternative 
projections  about  occurrences  in  the  environment  that  relate  to  seasonal 
changes.  It  is  quite  natural  for  the  leaves  to  change  color  in  accordance 
with  the  seasons  (at  least  in  Naw  Englandl ) .  However,  it  is  not  at  all 
natural  for  us  to  think  that  rocks  will  chbige  color  according  to  a 
seeisonal  pattern.  Analogously,  seasons  can  be  identified  by  the  names 
sunroer.  fall,  winter,  and  sprlm.  or  they  can  be  given  navel  names,  such 
as  soob.  trlt.  blen.  and  mave.  By  using  these  two  sets  of  concepts  and 
names,  the  following  four  situations  were  constructed:  (a)  familiar 
season  names  describing  states  of  the  leaves;  (b)  novel  season  names 
describing  states  of  the  leaves;  (c)  famiHar  season  names  d^cribing 
states  of  the  rocks;  and  (d)  unfamll iar  season  names  describing  states  of 
the  rocks.  In  the  second  experiment,  subjects  were  required  to  mate 
projections  about  events  in  the  envinzmaent  that  relate  to  periods  of  the 
day.  In  this  context,  it  is  natural  to  identify  a  period  of  the  day  by 
noting  the  position  of  the  sun  relative  to  the  horizon  and  it  is  quite 
unnatural  to  expect  that  minerals  will  change  shape  as  the  day 
progresses.  Also,  the  periods  of  the  day  can  be  Identified  by  the  names 
davtimie.  dusk,  nlbxttime.  and  dawn,  or  they  can  be  given  novel  names  such 
as  trofM:.  brai.  stobe.  and  kovit.  By  using  these  two  sets  of  concepts 
and  names,  a  set  of  four  situations  was  constricted  that  was  structurally 
equivalent  to  that  describing  seasons  in  the  first  experiment. 

Sidsjects  in  each  of  these  conditions  were  given  descriptions  of  the 
beginni^  and  end  of  a  season  (or  the  beginning  and  end  of  a  period  of  the 
day)  and  were  reqiiired  to  mate  Inferences  regarding  the  events  that 
occurred.  The  problans  were  presented  in  in^vidually  as  "selection  task 
items."  The  ease  with  vhich  subjects  made  these  judgments  was  measured  by 
both  latency  and  error  indices.  A  model  of  information  processing  was 
also  tested  for  the  latency  data  obtadned  in  each  of  the  four  tasks.  In 
addition  to  requiring  solu^on  of  selection-task  items,  these  esp<mdmients 
edso  required  subjects  to  solve  sets  of  reasoning  problems,  vhich  were  of 
varying  degrees  of  difficulty.  Mary  of  these  problems  were  similar  to 
ones  drawn  from  psychcmetric  intelligence  tests.  The  selection  task  items 
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therefore  ptrovided  a  way  to  asses  the  extent  to  ehlch  psychcnetrlcally 
measured  intelllgenoe  is  associated  with  tiie  ability  to  reason  within  new 
oonoeptual  systems. 

Models  of  Ncnentrenchment 

Ihese  eaqperiments  presented  an  opportunity  to  ocnpate  different 
stioictural  models  for  nonentrenched  oonoepts.  Ihe  potentled  effects  of 
lexical  familiarity  and  conceptual  naturalness  on  nonentrexicfament  can  be 
described  in  terms  of  five  basic  models,  which  are  shown  in  Table  29.  Ey 
evaluating  each  of  these  models,  we  can  determine  the  extent  to  which 
nonentrenchment  can  be  explained  in  terms  of  unf amlllarlty  or  interference 
from  prior  knowledge.  Analysls*-of*varlanoe  contrast  wei^its  are  used  to 
represent  the  patterns  of  reaction  time  and  error  rates  that  characterize 
each  of  these  models. 

Model  0 

In  Model  0,  the  null  case,  there  is  no  effect  for  either  lexical 
unfamlllarlty  or  oonceptual  tsmaturalness.  Ihls  model  that  the 

construct  of  nonentrenchment  Is  not  p^chologlcally  important.  In 
essence,  this  model  represents  the  null  hypothesis. 

Model  1 

In  Model  1,  the  locus  of  nonentrenchment  can  be  found  entirely  in 
conceptual  xmnaturalness;  according  to  this  fomnulatlon,  lexical 
familiarity  Is  not  a  source  of  interference, 
tfedgl  2 

Model  2  shows  the  oonplegoaentary  situation,  in  which  the  locus  of 
nonentrenchment  can  be  found  entirely  in  lexical  unfamiliarity;  according 
to  this  formulation,  oonceptual  naturalness  is  not  a  source  of 
interference. 


Ins«±  Table  29  about  here 


Model  3 

Model  3  describes  the  situation  in  which  lexical  unfamlliarlty  and 
conceptual  unnaturalness  both  contribute  to  nonentrenchment,  such  that 
their  effects  are  additive.  Ihis  model  distinguishes  betw^een  two  levels 
of  nonentrenchment.  On  one  level,  represented  by  "O"  in  the  table, 
ncnentrenchment  is  characterized  by  using  either  familiar  names  to  denote 
unnatural  occurrences  of  unfamiliar  names  to  denote  natural  occurrences. 

On  another  level,  there  is  a  more  difficult  form  of  nonentrenchment  that 
involves  vising  unfamiliar  names  to  denote  unnatural  occurrences.  This 
model  therefore  involves  both  unfamiliarity  and  unnaturalness  contributing 
additively  to  nonentrenchment.  Moreover,  if  we  assume  that  vmnatural 
concepts  are  in  sene  sense  unfamiliar,  this  model  also  indicates  that  the 
overall  amount  of  unfamiliarity  in  a  ocnoeptual  system  defines  the  extent 
to  which  that  ooncfptual  system  is  nonentrenched. 

Modal  4 

Finally,  Model  4  describes  the  situation  in  which  nonentrencdnient  is 
defined  by  an  interaction  between  lexical  unfamiliarity  and  conoeptued 
vmnatuialness.  Aexxording  to  this  model,  there  are  two  equivalent  types  of 
nonentrenched  conceits,  one  in  which  familiar  names  denote  unnatural 
occurrences,  and  another  in  which  unfamiliar  names  denote  natural 
occurrences.  Nonentrenchment  is  the  result  of  an  interference  effect,  in 
which  the  interference  is  generated  by  either  a  familiar  name  or  a  natural 
concept,  but  not  both  simultaneously.  Ihis  model  therefore  shows  that 
nonentrenchment  is  distinct  from  the  cverall  vmfamiliarlty  of  a  given 
conceptual  system. 
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E:q)earlment  1 
Method 

Subjects.  Subjects  were  96  Yede  laidergraduates  participated  for 
course  credit,  ncnetary  payment,  or  both.  Subjects  were  randcmly  assigned 
to  one  of  four  oonditicns,  with  24  suln^ects  in  each  condition.  A  separate 
group  of  25  undergraduates  taking  a  develcpnent2d  psychology  course  at 
Yede  gave  responses  to  a  set  of  background  survey  questions. 

Materials.  Uie  basic  materials  were  selection^task  items  presented 
via  a  tachistcsoope  and  psychcmetric  inductive  and  deductive  ability  tests 
presented  in  a  paper^-and-pencil  format,  m  addition  to  the  standardized 
ability  tests,  subjects  were  given  a  set  of  problems  that  have  previously 
been  used  to  study  insist  (Sternberg  &  Davidson,  1982) . 

The  selectiovtask  items  were  modeled  on  previous  problems  used  by 
Sternberg  (1982) .  Items  were  developed  in  which  a  common  "deep"  structure 
was  used  to  generate  four  different  "surface”  structures.  In  each  of  the 
four  sets  of  items,  the  problems  were  based  on  the  initial  premise  that 
the  seasons  of  the  year  allow  one  to  predict  oertadn  occurrences  in  nature 
and  that,  in  turn,  these  occurrences  identify  what  a  given  season  is. 

Each  problem  contained  two  pieces  of  information.  The  first  piece  of 
information  described  a  situation  at  the  beginning  of  a  season  and  the 
second  piece  of  information  provided  follow-up  data  from  the  end  of  the 
same  season.  Because  each  of  the  four  variaticns  of  the  task  were 
similar,  only  one  version  will  be  described  in  detail.  The  other 
variations  vdll  be  desorbed  more  briefly. 

The  premise  that  served  as  the  model  for  the  other  three  versions  of 
the  task  stated  that  in  New  Haven,  the  beginning  and  end  of  each  season 
are  marked  by  the  fact  that  the  leaves  will  be  either  green  or  brcwn.  In 
summer,  the  leaves  are  green  at  the  beginning  and  at  the  end  of  the 
season.  In  fadl,  the  leaves  are  green  at  the  beginning  of  the  season  but 
are  brown  at  the  end.  In  winter,  the  leaves  are  brown  at  both  the 
beginning  and  end  of  the  season.  And,  in  apring,  the  leaves  are  brown  at 
the  beginning  of  the  season  but  are  green  at  the  end.  Subjects  were 
required  to  use  this  information  to  solve  a  series  of  reasoning  problems. 

Each  problem  was  presented  on  one  card.  Each  term  of  a  problem  could 
oontaiin  one  of  two  forms  of  information.  The  descripticn  could  be  either 
a  picture  of  the  leaves,  indicated  by  a  green  or  brown  circle,  or  the  name 
of  a  season  that  represents  a  decision  about  what  season  it  is,  based  on 
the  color  of  the  leaves  at  the  time  the  observation  was  made.  Information 
about  the  leaves  at  the  beginning  of  the  season  appeared  on  the  left  and 
information  about  the  leaves  at  the  end  of  the  season  appeared  on  the 
ri^t.  Because  each  of  tws  descriptions  of  the  leaves  (one  at  the 
beginning  of  the  season  and  one  at  the  end)  could  take  ^ther  of  two 
physical  forms  (brown  or  green)  or  four  verbal  forms  (an  inference  based 
on  a  season  name) ,  there  were  6  X  6,  or  36,  distinct  items. 

The  subject's  task  was  to  describe  the  leaves  at  the  end  of  the 
season,  based  on  the  information  provided  in  the  problem.  If  the  given 
description  for  the  end  of  the  season  was  a  picture  of  the  leaves,  the 
subject  has  to  indicate  the  correct  name  of  the  season.  If  the  given 
description  for  the  end  of  the  season  was  a  name,  the  subject  had  to 
indicate  the  correct  color  of  the  leaves.  There  were  edways  three  answer 
choices,  from  vMch  the  subject  had  to  choose  the  correct  one.  These 
edtematives  appeared  below  the  problem  stem. 

There  were  four  different  types  of  problems.  Items  either  had  two 
season  names,  a  picture  followed  by  a  season  name,  a  season  name  follcwed 
by  a  picture,  or  two  pictures.  In  the  first  two  types  of  problems, 
subjects  had  to  determine  the  color  of  the  leaves  at  the  end  of  the 
season — in  the  other  two  problems,  subjects  has  to  give  the  name  of  the 
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season  consistent  with  the  given  Information.  Subjects  were  ederted  to  a 
further  conplexity  in  the  selection  teu^,  which  also  applies  in  the  real 
world.  At  the  beginning  of  a  season.  It  is  Ispossible  to  distinguish 
summer  fron  fall  or  firing  from  wint^.  If  the  only  etvedlable  information 
Is  the  initial  color  of  the  leaves.  Also,  the  names  "sunner"  and  'Vinter" 
imply  that  the  leaves  will  remain  the  same  color,  vhereas  the  names 
"spring"  and  "fall"  imply  that  the  leaves  will  change  color  by  the  end  of 
the  season.  For  the  problems  in  which  the  first  term  was  the  name  of  the 
season,  this  name  oorxectly  described  the  color  of  the  leaves  at  the 
beginning  of  the  season,  but  only  predicted  what  color  the  leaves  would  be 
at  the  end.  Ihls  prediction  ml^t  not  correspond  to  the  color  that  was 
described  by  the  second  term.  Thus,  It  was  not  possible  to  know  for 
certain  the  color  of  the  leaves  at  the  end  of  the  season  or  to  know  for 
oert2dn  the  tnie  season.  When  the  first  term  of  the  problem  was  a  picture 
of  the  leaves,  this  ocnplexlty  did  not  exist,  because  a  physlced 
description  carries  no  implication  regarding  the  future  physical 
appearance  of  the  leaves. 

Althou^  this  uncertainty  in  prediction  did  not  exist  for  information 
describing  the  leaves  at  the  end  of  the  season,  there  was  a  related 
problem  associated  with  the  second  term.  When  a  season  name  described  the 
leaves  at  the  aid  of  the  season,  it  could  be  assanned  to  provide  correct 
infcnmoatlon  about  both  the  beginning  and  the  ending  color  of  the  leaves, 
because  assessments  of  the  season  made  late  in  the  season  were  based  on 
observations  of  the  leaves  throughout  the  entire  season.  For  the  problems 
In  which  the  second  term  was  a  name,  however,  this  season  name  could  be 
"inconsistent"  with  the  starting  color  of  the  leaves,  as  defined  in  the 
first  term  of  the  problem.  For  exanple,  if  the  first  term  were  "summer, " 
this  name  means  that  the  leaves  were  green  at  the  beginning  of  the  season 
and  predicts  that  they  would  be  green  at  the  end.  If  the  second  term  were 
"spring,"  this  name  means  that  the  leaves  were  brown  at  the  start  of  the 
season  and  eventually  turned  green.  Because  the  leaves  cannot  be  both 
brown  and  green  at  the  beginning  of  the  season,  this  problem  describes  an 
Inocnslstent  situation;  as  a  result.  It  was  im(»6sible  to  determine  the 
color  of  the  leaves  at  the  end  of  the  season.  Ihe  correct  answer  was  thus 
"inconsistent." 

To  sunmarlze,  physical  descriptions,  which  carried  no  necessarily 
oorrect  inpllcatlon  for  vhat  the  leecves  would  look  like  at  another  time, 
were  eLLways  accurate  with  respect  to  the  appearance  of  the  leaves  at  the 
time  of  the  description.  However,  they  mi^t  not  be  accurate  with  respect 
to  the  eppearanoe  of  the  leaves  at  the  end  of  the  season.  A  complete 
listing  of  the  36  distinct  problems  can  be  found  in  the  i^pendix. 

Ihls  esq^eriment  attempted  to  assess  the  extent  to  vhlch  various 
oonoeptual  systems  are  more  or  less  "entnmiched"  by  ocnparing  hew 
different  problem  contents  and  forms  affect  reasoning.  In  the  form 
mentioned  above,  subjects  were  required  to  reason  within  an  entrmuched 
framework.  Ihe  other  three  forxas  of  this  task  varied  either  the 
naturalness  used,  or  both.  It  Is  expected  that  leaves  will  alternate 
between  green  and  brewn  as  the  seasons  change.  Hewever,  It  is  not  at  all 
normal  to  esqiect  that  rocks  will  change  frem  orange  to  blue  with  the 
passage  of  seasons.  Similarly,  the  terms  summer,  fedl,  winter,  and  ^ring 
carry  certain  connotations  about  the  seasons  they  name,  but  the  neologisms 
soob,  trlt,  blen,  and  mave  do  not  carry  any  uneg^voced  Information  about 
the  physic^  world.  Because  there  are  two  types  of  concepts  (natural  and 
unnatural)  and  two  types  of  names  (familiar  and  novel) ,  there  are  2  X  2  or 
four  possible  versions  of  the  season-color  information. 
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In  a  secxnd  oondlticn,  subjects  vere  told  about  the  distant  country  of 
Latzanla,  viiere  the  leacves  change  color  jvist  as  they  do  in  New  Haven,  but 
the  seasons  are  called  trit,  blen,  nave,  and  scab.  In  a  third  oanditicn, 
subjects  were  told  about  the  planet  ^rm,  \diere  the  seasons  are  called 
summer,  fall,  winter,  and  spring,  but  are  narioed  by  the  fact  that  rocks 
change  frcm  blue  to  orange  or  frca  orange  to  blue.  In  the  fourth 
oanditicn,  subjects  were  told  about  the  planet  I^rcn,  where  the  seasons 
are  called  trit,  blen,  macve,  and  socb,  and  can  be  distinguished  by  the 
fact  that  the  rocks  vary  fron  orange  to  blue,  aooording  to  a  systematic 
pattern.  Each  of  the  four  tasks  is  sunoariaed  in  Table  30. 


Insert  Table  30  about  here 


Using  each  of  the  conditions  described  above,  three  more  sets  of  36 
problems  were  generated  that  were  structuredly  identical  to  those 
described  for  the  case  of  New  Haven.  The  only  difference  amooig  the  four 
tasks  involved  the  surface  structure  of  the  items,  as  defined  by  a 
particular  concept — language  ocobination.  This  nsuiipulaticn  of  content 
made  it  possible  to  identify  the  locus  of  nonentrenchment. 

The  ability  tests  used  in  the  experiment  were  geccstric  series 
ccnpletions  (Abstract  Reasoning)  from  the  Differaitial  Aptitude  Test 
(Bennett,  Seashore,  &  Wesman,  1973) ,  letter  and  nunber  series  ocnpletions 
(Reasoning)  fron  the  SRA  Primary  Hent2d  Abilities,  adult  level  (Thurstone, 
1962) ,  and  deductive  syllogisms  (subtest  3)  and  confirming  the  validity  of 
conclusions  (subtest  9)  frca  the  Watson-Glaser  (1964)  Critical  Thinking 
Appredsal. 

The  surv^  requested  subjects  to  rate  hew  common  it  is,  in  their 
experience,  fca:  either  leaves  or  rocks  to  change  colors  with  the  seasons. 
In  addition,  subjects  were  given  an  array  with  the  names  "summer,”  "fall," 
'%dnter,"  and  "spring"  on  one  side  and  the  four  possible  blue-orange  or 
brown-green  pairings  on  the  other  side.  Their  ta^  was  to  match  one 
season  name  with  one  of  the  four  physical  occurrences.  Subjects  answered 
questions  about  either  leaves  or  rocks,  but  not  both. 

Design.  The  overall  design  placed  subjects  within  a  2  X  2 
between-subjects  factorial  arrangemait.  The  primary  dependent  variable 
was  solution  latency;  a  secondary  dependent  variable  was  error  rate.  In 
item  construction,  independent  variables  were  the  six  possible  state 
descriptions  [  (e.g. ,  in  the  case  of  New  Haven,  summer,  fedl,  winter, 
spring,  6  (green  ciixde) ,  and  B  (brown  circle)  crossed  with  the  two 
possible  times  of  occurrence  (the  beginning  and  end  of  a  season)  ] .  All 
subjects  saw  each  of  the  36  possible  item  types  three  times,  with  the 
ooinrect  option  in  a  different  location  each  time.  The  items  in  each  task 
variant  were  groeped  into  three  blocks  so  that  a  subject  had  to  ocnplete 
an  entire  set  of  36  items  before  seeing  an  item  for  a  subsequent  time. 

Each  subject  was  randomly  assigned  to  one  of  the  four  versions  of  the 
selection  ta^.  For  items  in  which  subjects  had  to  determine  the  color  of 
the  leaves  at  the  end  of  the  season,  distractors  consisted  of  an  incorrect 
picture  and  an  "indeterminate"  (I)  option,  to  correspond  to  the 
possibility  that  the  information  in  the  problem  could  be  describing  a 
self-contradictory  and  hence,  indeterminate  situation.  For  the  items  that 
required  subjects  to  determine  the  name  of  the  season,  distractems 
consisted  of  two  of  the  three  possible  word  distractors  bedanced  over  the 
three  replications  of  the  ta^  for  a  given  esq^riment.  Thus,  each 
possible  word  distractor  appeared  equally  often  across  item  replications. 
In  the  three  versions  that  described  the  seasons  in  latzania  or  on  lyron, 
four  counterbalanced  forms  of  the  items  were  constructed  so  that  each 
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season  name  was  pedred  with  each  of  the  four  pt^ical  oocurvenoes  (two 
oonoepts  describing  a  physicad.  change  and  two  oonoepts  describing  ocnstant 
physical  states)  only  once.  For  exarple,  in  latzania,  the  season  that 
would  correspond  to  sumtner  was  socb  in  Form  A,  trit  in  Form  B,  blen  in 
Form  C,  and  nave  in  Form  0.  Ihis  scheme  was  foUcwed  for  each  of  the 
other  three  seasons.  Ihis  method  of  counterbedancing  was  not  applied  to 
the  Ifew  Haven  scenario,  hcwever,  because  this  scenario  did  not  have  any 
possible  edtemative  forms.  Subjects  who  filled  out  the  survey  were  given 
one  of  two  alternative  forms  of  the  questions.  The  forms  differed  in  the 
order  that  the  season  names  were  listed,  so  that  subjects  would  not  be 
biased  in  favor  of  choosing  a  particular  season  name  for  ai^  one  color 
change. 

Apparatus.  Selection-task  items  were  administered  on  an  Iconix 
tachlstoscope  with  attached  millisecond  timer.  Subjects  signaled  their 
responses  by  pressing  the  button  corresponding  to  the  a^^propriate  answer 
option.  Psychometric  ability  tests,  insight  problems,  and  the  survi^  were 
2dl  administered  in  written  form. 

Procedure.  Subjects  conpleted  an  informed  consent  from  and  then  were 
given  instrxictions  for  the  appropriate  task.  Ihe  instructions  were  rather 
lengthy  and  required  subjects  learn  hew  to  solve  an  entirely  new  set  of 
reasoning  problems.  Because  of  the  ccnplexity  of  the  task,  after  each 
subject  finished  reading  the  instructions,  the  e^qserimenter  reviewed  the 
essentied  elements  involved  in  each  of  the  four  types  of  problems.  Ihen 
subjects  received  ei^t  practice  items,  two  of  each  of  the  four  types  of 
problems  described  earlier.  Vlhen  needed,  extra  practice  items  were 
provided  tmtil  subjects  were  able  to  give  correct  responses  and 
demonstrate  that  th^  were  aware  of  the  different  reepirements  of  each 
problem.  Subjects  were  instructed  to  solve  the  items  as  rapidly  as  they 
could  under  the  constraint  that  they  be  as  accurate  as  possible.  After 
the  practice  trials  were  over,  subjects  received  three  randomized  blocks 
of  36  problems,  each  of  a  different  type.  Each  of  the  36  problem  ^pes 
appeared  once  in  each  of  the  three  blocks.  Items  were  dra^  on  separate  6 
X  9-in.  cards.  Each  answer  c^ion  was  correct  equedly  often  in  each  block 
(12  times  per  block) .  The  e^q^erimenter  initiated  each  trial.  Ihe 
millisecond  clock  started  as  soon  as  the  stimulus  was  Illuminated  in  the 
tachistoscepe.  It  stopped  as  soon  as  the  subject  pressed  one  of  the  three 
answer  buttons.  In  general,  feedback  was  not  provided  during 
selection-task  trials,  unless  subjects  made  three  errors  in  a  row.  Ihis 
feedback  was  given  to  ensure  that  subjects  were  aware  of  the  various 
intricacies  involved  in  the  different  types  of  problems.  Of  the  96 
subjects  vho  participated  in  this  e^q^eriment,  12  were  given  feedback.  Ihe 
selection  task  losually  took  about  l  h.  to  administer. 

Ihe  ctbility  tests  were  administered  at  a  later  time  in  anail  groips. 
All  tests  were  timed  and  subjects  were  told  to  conplete  the  tests  as 
quickly  and  accurately  as  possible.  Ihe  various  tests  were  edways 
presented  in  the  following  order,  \nder  the  specified  time  constraints: 

(1)  letter  series  (2  1/2  min) ,  (2)  deductive  syllogisms  (6  min) ,  (3) 
number  series  (2  1/2  min) ,  (4)  confirming  the  validity  of  oonclxjsicns  (4 
min) ,  (5)  abstract  reasoning  (12  min) ,  and  (6)  insist  problems  (20  min) . 

InformaticMi-Processing  Model 

A  proposed  information-processing  model  for  performance  in  the 
selection  task  is  shewn  in  Figure  5.  Information  processing  is  assumed  to 
be  serial  with  respect  to  the  component  processes  in  the  model.  Each 
process  is  assumed  to  contribute  additively  to  total  solution  latency,  and 
each  process  was  assigned  a  mathematical  parameter  to  represent  its 
duration.  Ihis  model  is  similar,  but  not  identical,  to  the  model  used  in 
Sternberg  (1982).^ 


Insert  Figure  5  about  here 
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Basic  Statistics 

Main  aameHTwanfe.  Basic  statistics  for  the  groups  defined  by  the  2X2 
design  are  provided  in  Table  31.  Each  solixticn  time  is  based  cn  108 
cbservaticns  per  subject,  averaged  over  the  24  subjects  in  each  group.  An 


Insert  I^le  31  about  here 


analysis  of  variance  was  done  on  the  reaction  time  data.  liie  rain  effect 
of  concept  (leaves  turning  brown  ard  green  versus  rodks  turning  orange  and 
blue)  was  not  significant,  F(l,92)  -  1.21,  p  >  .05.  Hie  main  effect  of 
language  (coomcn  or  novel  season  names)  edso  was  not  significant,  F(1.92) 
a  1.87,  p  >  .05.  However,  the  concept  X  language  interaction  was  hi^ily 
significant,  F(l,92)  »  7.64,  p  <  .01.  Hie  analysis  of  variance  on  the 
error  data  yielded  the  same  pattern  of  results.  Hie  main  effects  of 
concept  and  language  fadled  to  reach  significance,  F(l,91)  -  .03  and 
F(1.92)  -  1.76  (p  >  .05  in  each  case),  respectively.  The  interaction 
between  these  two  effects  was  significant,  F(l,92)  -  11.00,  p  <  .01. 

Hiese  results  show  that  the  items  in  sets  2  and  3  were  the  most  difficult 
to  reason  with  because  they  produced  the  longest  latencies  and  caused 
subjects  to  make  the  most  errors.  Hius,  as  described  by  Model  4,  a 
nonentrenched  concept  takes  one  of  two  forms:  It  may  require  us  to 
identify  a  natural  occurrence  with  an  unfamiliar  name,  or  it  may  require 
us  to  use  familiar  names  to  describe  occurrences  that  are  umatnred.  If  a 
concept  uses  unfamiliar  names  to  describe  unnatural  occurrences,  it  is 
easy  to  reason  with  because  it  is  not  subject  to  interference  from  prior 
knowledge. 

Hie  correlations  between  latencies  and  exrcac  rates  ocoputed  across  24 
Aibjects  argue  against  a  Epeed-aocuracy  txadeoff~.25,  .47,  .44,  and  .25 
for  sets  1,  2,  3,  and  4,  respectively.  These  positive  correlations 
suggest  that  subjects  with  longer  latencies  tended  to  make  more  errors. 
Only  the  correlations  for  sets  2  and  3  were  significant,  p  <  .05, 
two-tadled. 

Oorrelations  between  error  rates  axid  lantencies  across  the  36  item 
types  were  substantial,  in  addition  to  being  hi^ily  significant — .64,  .69, 
.61,  and  .45  for  sets  1,  2,  3,  and  4,  respectively.  Hie  first  three  were 
significant  at  the  .0001  level  and  the  fourth  was  significant  at  the  .01 
level,  using  two-tEdled  tests.  Hie  fact  that  subjects  made  more  errors  on 
items  with  longer  latencies  may  mean  that  when  more  processing  operations 
are  required,  subjects'  reasoning  capacity  is  more  likely  to  become 
overloaded,  resulting  in  incorrect  solutions. 

Because  the  two  dependent  variables  were  hi^ily  correlated,  a 
multivariate  analysis  of  variance  was  performed.  Hiis  analysis  replicated 
the  results  fron  the  two  univaudate  analyses  of  variance.  For  the  concept 
madn  effect,  Wilks  .986,  F(2,91)  *  .63,  p  >  .05.  For  the  language 
main  effect,  Wilks  «  .972,  F(2,91)  «  1.31,  p  >  .05.  Only  the 

interaction  was  significant:  Wilks  /I  .870,  F(l,92)  *  6.79,  p  <  .01.  A 
foUcw^^  discriminant  analysis  cn  the  interauTtion  was  carried  cut  to 
determine  the  extent  to  '»diich  each  of  the  two  dqmndent  variables  was 
irportant  in  determining  this  effect.  Hie  resulting  discriminant  functicn 
was  significant—Wilks  /v  -  .874,  X^  (2,  N  =  48)  ■  12.55,  p  <  .05.  Hie 
standardized  discriminant  function  weights  were  .469  for  latency  and  .721 
for  errtm:  rates.  Hie  latency  and  error  scores  were  hi^ily  correlated  with 
the  discriminant  function,  .75  and  .90,  respectively.  Hiese  anadyses 
confirm  that  the  conceit  X  language  interac^on  was  not  an  artifact  due  to 
the  correlation  between  d^iendent  measures.  Both  the  latency  and  error 
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data  sippart  the  claim  that  the  locus  of  ncneritrendhment  lies  in  ocmbining 
scnething  familiar  with  scmething  that  is  novel.  Ihe  discriminant 
analysis  also  shows  that  both  of  the  dependent  measures  provide 
oasplementaiy  informatian  in  identifying  differences  between  the  groups. 

Survey.  The  survey  was  oonducted  to  provide  supporting  evidence  that 
within  this  pcpulaticn,  certain  occurrences  were  more  natural  than 
others.  Hie  frei^iencies  with  \diich  subjects  described  the  seasons  summer, 
fall,  winter,  and  spring,  as  being  related  to  a  particular  color  change, 
were  tabulated.  In  the  first  survey,  91%  of  the  responses  were  in 
acoordanoe  with  the  predicted  relatiaiship;  that  is,  leaves  are  green 
throue^iout  sunoaer,  in  fall  they  start  out  green  and  eventually  turn  brown, 
they  are  brown  throuc^icup  wlnti^,  and  in  spring  they  start  out  brown  and 
eventually  turn  green,  (1,  N  «  11)  »  189.1,  p  <  .001.  In  the  second 
survey,  althou^  subjects  agreed  that  it  was  not  at  edl  ocrnimnn  for  rodcs 
to  change  color  with  the  seasons,  they  inferred  that  one  pattern  of 
changes  was  more  reasonable  than  any  other;  79%  of  the  responses  were  in 
line  with  the  idea  that  in  summer  rocto  should  be  orange  thrcughout  the 
entire  season,  in  fall  they  should  be  orange  at  the  beginning  and  blue  at 
the  end  of  the  season,  in  winter  they  should  be  blue  throu^iout  the  entire 
season,  and  in  spring  th^  should  be  blue  at  the  beginning  of  the  season 
and  orange  at  the  end,  Xr  (1,  N  «  14)  •  188.4,  p  <  .001. 

The  results  of  the  f  ir^  survey  were  in  acoordanoe  with  the  latency 
and  error  data  described  above;  the  conceptual  system  in  set  1  was  rated 
as  being  very  familiar,  and  it  was  also  associated  with  duorter  solution 
latencies  and  with  lower  error  rates.  However,  the  second  survey 
predicted  that  one  form  of  set  3  could  be  inteipreted  in  a  meaningful 
way.  Each  of  sets  2,  3,  and  4  was  divided  into  four  counterbalanced 
versians~-A,  B,  C,  a:^  I>~wlth  six  subjects  seeing  each  version.  Ihe  mean 
latencies  for  the  subsets  in  set  3  were  4.83  ,  5.02  ,  4.74,  and  4.61  s, 
respectively.  Subjects  in  the  survey  singled  out  3C  as  being  most 
interpretable.  In  this  versicn,  rocdcs  are  orange  throu^iout  summer,  in 
fall  they  start  out  orange  and  eventually  turn  blue,  they  are  blue 
throughout  winter,  and  in  firing  they  start  out  blue  and  eventually  turn 
orange.  An  analysis  of  variance  on  the  respective  times  for  the  four 
versions  of  set  3  indicated  that  they  were  not  significantly  different, 
however,  F(3,20)  «  .06,  p  >  .05.  Hxus,  it  appears  that  ccytoeptua]  ^stem 
3C  was  not  easier  to  reason  with.  Nhen  forced  to  think  about  the  seasons 
in  terms  of  rocdcs  turning  orange  and  blue,  subjects  seem  to  associate 
winter  with  the  colder  ooloi>H3lue-*-and  summer  with  the  warmer 
color — orange.  However,  this  information  does  not  seem  to  eiffect 
latencies  in  the  information-processing  task  used  in  this  esperiment. 
onant-i-hative  Tests  of  the  mformation-Processino  Model 

Table  32  shows  the  results  of  various  statistical  evaluations  of  the 
information-processing  model  of  selection-task  performance.  Hiese  results 
are  based  only  on  items  that  were  answered  correctly.  However,  the  model 
was  also  tested  on  data  frcm  the  ocnbined  correct  and  inoranject  items. 
There  were  no  noteworthy  differences  between  the  two  sets  of  armlyses. 


Insert  Table  32  about  here 


The  model  provides  a  good  fit  to  the  latency  data  and  is  consistent 
across  all  four  instantiations  of  the  task.  The  values  of  Ir 
(proportion  of  variance  accounted  for  in  the  latency  data  by  the  model) 
are  all  in  the  low  .90s.  The  item  reliabilities  are  all  in  the  mid  to 


hi^  .90s,  meaning  that  the  various  items  are  consistent  in  the  processes 
th^  are  assessing.  These  model  fits  are  highly  significant  (with  respect 
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to  the  difference  of  R?  fran  0) ,  althou^  the  residuals  were  also 
significant.  The  model  edso  provides  a  very  good  fit  to  individu2d. 
subjects'  data,  with  Ir  values  for  individu^  subjects  ranging  frcm  .24 
to  .91  across  the  four  data  sets.  Ihe  mean  individual  ir  values  for 
each  of  the  four  groi^  were  .67,  .70,  .69,  and  .71,  for  grot^s  1,  2,  3, 
and  4,  respectively.  The  root-nean-sguare  deviaticn  of  observed  frcm 
predicted  latencies  for  individual  sul^ects  ranged  from  .31  to  .45,  vhioh 
is  quite  good  given  that  the  grand  mean  for  all  items  was  4.14  s.  The 
original  model  developed  fay  Sternberg  (1982)  provid^  good  fits  vhen 
applied  to  the  data  sets  in  this  eaqperiment.  ’Bob  It  values  for  sets  1, 

2,  3,  and  4  were  .92,  .90,  .91,  and  .89,  reflectively.  However,  the 
revised  model  represents  a  statistically  sicpiif leant  inprovement  in  terms 
of  the  parameter  estimates.  (Other  models  were  tested  with  less 
satisfactory  results.) 

Parameter  Estimates  for  the  Information-Procegsiner  Mortal 

Ihe  regression  model  provides  parameter  estimates  that  describe  how 
long  each  of  the  processing  steps  takes  for  each  version  of  the  task.  The 
parameter  estimates  (raw  regression  wei^xts)  are  presented  in  Table  33. 

All  five  parameters  were  significantly  different  from  zero  across  each  of 
the  four  tasks.  They  also  followed  the  same  pattern  of  relative 
difficulty,  except  for  assess  Initial  information  (All)  and  change  in 
representational  system  (CRS)  in  set  3.  Thus,  the  model  is  measuring 
processes  that  are  general  across  several  tasks. 


Insert  Table  33  about  here 


The  estimates  in  sets  2  and  3  were  generally  longer  than  their 
ocunterparts  in  the  other  two  sets,  the  longer  estimates  in  these  two 
sets  correspond  to  the  fact  that  these  two  conditions  were  more  difficult 
to  reason  with,  as  shewn  in  the  analysis  of  variance.  They  also  suggest 
that  the  effects  of  nonentrenchment  are  localized  in  particular  exmponent 
processes.  For  exzmple,  the  justification  (JUS)  parameter  showed  the 
largest  increase  in  waight,  suggesting  that  subjects  find  it  especiedly 
difficult  to  evaluate  the  vall^ty  of  their  solution  when  they  have  to  use 
a  familiar  name  that  describes  an  unnatural  occurrence  or  an  unfamiliar 
name  that  describes  a  natural  ocxurrence.  The  change  in  physical  state 
(CPS)  parameter  had  the  next  largest  increase  that  M^it  be  attributable 
to  nonentrenchment.  This  parameter  reflects  the  faext  that  variable-state 
concepts  involve  two  colors  that  occur  in  a  ^^ecific  sequence,  vhereas 
concepts  that  describe  a  constancy  contedn  only  one  color  and  so  subjects 
do  not  have  to  deal  with  the  order  of  the  two  colors  to  know  which 
constant  season  is  being  referenced.  Thus,  in  terms  of  processing 
effects,  nonentrenchment  may  make  it  more  difficult  to  store  or  access  the 
information  in  these  variable-state  concepts.  The  fact  that  uneiqiected 
lack  of  change  (UCVS)  and  CBS  are  large  in  set  3,  but  not  in  set  2, 
suggests  that  these  parameters  reflect  the  difficulty  involved  in 
processing  the  pictures  used  to  represent  the  unnatured.  concepts. 
Similarly,  AH  is  large  in  set  2  but  not  in  set  3,  iaplying  that  this 
parameter  is  eiffected  mainly  by  lexiced  familiarity.  Overall,  the 
parameters  seem  to  reflect  processes  in  the  reasoning  task  that  are 
subject  to  interference  frcm  pairing  something  familiar  with  something 
unfamiliar. 

Tndividuai  Differences  Analysis 

Scores  on  the  three  inductive  tests  were  standardized  and  added 
together  to  obt2dn  an  overall  score  for  inductive  reasoning  ability.  The 
same  type  of  oenputation  was  done  to  oembine  scores  for  the  two  deductive 
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reasoning  tests.  Itiis  fooi  of  anedysls  was  done  because  the  ocnoposite 
scores  hetve  greater  reliability  than  the  Individual  test  scares.  These 
ability  measures,  in  addition  to  ttie  insi^it  poxhlens,  were  oorrelated 
with  each  subjects's  mean  latency,  averaged  over  108  Itans.  Parameter 
scares,  obtained  by  fitting  the  infarmation-processing  model  to  individual 
subjects'  latency  data,  were  also  oorrelated  with  tiM  ability  tests.  The 
correlations  eu:e  shown  in  Table  34.  (In  these  anadyses,  the  tests  were 
standardized  by  treating  each  of  the  four  gcaape  separately.  When  the 
tests  were  standardized  without  regard  to  group  membership,  the  results 
were  vlrtaally  identical.) 


Insert  Table  34  about  here 


Consider  first  the  correlations  with  the  inductive  measure.  The 
oorrelatians  of  the  global  latencies  with  the  inductive  reeusoning 
composite  are  the  highest.  However,  th^  are  not  consistently  hi^  for 
each  of  the  four  grocps—groip  2  was  the  exception  with  a  oorrelation  of 
-.14.  Adding  in  error  rate  as  a  second  predictor  variable  did  not  change 
this  pattern  of  results— the  multiple  oarxelations  were  -.63,  -.15,  -.57, 
and  -.40,  for  grocps  1,  2,  3,  and  4,  respectively.  The  correlations  of 
parameter  scores  vdth  the  Inductive  reasoiing  measure  essentially  follow 
this  same  pattern,  with  group  2  fedling  to  show  significant  correlations. 

Oonsidi^  next  the  correlation  of  the  global  task  and  parameter  scares 
with  the  deduction  measure.  The  global  task  shows  a  pattern  of 
oorxelations  that  is  analogous  to  that  found  with  the  inductive  reasoning 
measure;  however,  the  correlations  with  deductive  ability  are  consistently 
lower.  This  same  basic  pattern  occurs  with  the  task  score  correlations. 

The  insist  prcblenis  were  hi^y  intercorrelated  with  both  the 
inductive  and  deductive  test  scores.  The  correlations,  averaged  over  the 
four  grotps,  were  .63  with  induction  and  .44  with  dedu^on.  This 
difference  in  correlations  implies  that  the  insist  problems  are  more 
related  to  induction— the  ability  to  take  the  information  stated  in  a 
problem  and  to  form  a  conclusion  that  is  not  directly  related  to  the  given 
information.  In  terms  of  the  selection  ta^,  the  overall  response  latenoy 
for  set  3  had,  by  fsar,  the  hic^iest  oorrelatim  with  the  insic^  problems. 
Moreover,  the  Inductive  scores  for  set  3  had  more  significant  correlations 
with  task  parameters  than  did  any  of  the  other  three  sets.  Because  the 
items  in  set  3  had  familiar  names  mapped  onto  novel  concepts,  these 
results  suggest  that  insi^it  is  closely  related  to  the  ability  to  vise 
existing  information  in  the  lexicon  to  describe  vmfamiliar  occurrences  in 
the  environment.  Insic^  can  therefore  be  viewed  as  a  specialized  form  of 
Inducing  structure  in  problem  situations. 

Althcu^  this  e)qp^±nait  yielded  results  that  were  both  reliable  and 
interpretable,  it  is  possible  that  nonentrenchment  is  ^lecific  to  this 
particular  experimental  manipulation.  Berlin  and  Kay  (1969)  have  shown 
that  there  is  something  basic  about  the  way  our  exp&ri&nsB  with  colors  is 
coded  in  language.  Their  work  suggests  that  the  present  set  of  results 
may  be  attributable,  at  least  in  part,  to  the  fact  that  the  seasons  were 
defined  in  terms  of  colors.  It  is  oorx:eivable  that  a  conc^  that  is 
described  by  something  other  than  color  may  have  different  properties 
within  this  experimental  design.  Because  of  this  possibility,  a  second 
eiperlment  was  carried  out  to  show  that  the  language  X  oono^Tt  interaction 
can  be  demonstrated  for  other  kinds  of  conceptual  systems. 

Experiment  2 

This  eaperiment  was  intended  to  replicate  and  generalize  the  findings 
of  the  first  stuty.  The  major  differences  involved  the  content  of  the 
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ocnoeptual  system  and  the  words  used  to  describe  this  ocntent.  Iherefore, 
the  description  of  this  es^ieriment  will  focus  only  on  those  aspects  of  the 
method  that  differed  from  that  \]sed  in  the  first  esperiment.  The  anedysis 
of  the  results  directly  parallels  the  form  used  in  the  first  es^eriment. 

Method 

Subjects.  Subjects  were  80  Yale  undergraduates  who  participated  for 
course  credit,  monetary  payment,  or  both.  Subjects  were  randomly  assigned 
to  one  of  the  four  conations,  with  20  subjects  in  each  condition.  It 
should  be  noted  that  5  subjects  were  replaced  because  they  had  error  rates 
that  were  near  or  above  the  30%  level.  A  separate  grocp  of  24 
undergraduates  who  were  participating  in  other  peychology  experiments  gave 
responses  to  a  set  of  badoground  survey  questions. 

Materials.  The  selection-task  items  used  in  this  eageriment  were 
structurally  identical  to  those  used  previously — they  differed  only  in 
content.  The  premise  that  served  as  the  model  for  the  other  three 
versions  of  the  task  stated  that  in  Mew  Haven,  the  beginning  and  end  of 
each  period  of  the  day  is  marked  by  the  fact  that  the  sun  is  either  above 
or  below  the  horizon.  At  dawn,  the  sun  starts  out  below  the  horizon  but 
eventually  ends  \;p  above  the  horizcn.  In  daytime,  the  sun  is  above  the 
horizon  at  the  beginning  and  at  the  end  of  this  period.  At  dusk,  the  sun 
starts  out  above  the  ho^on  but  eventuedly  ends  below  the  hax±son. 

And  in  ni^ttime,  the  sun  is  below  the  horizon  at  the  beginning  and  at  the 
end  of  this  period. 

To  parallel  the  first  experiment,  the  periods  of  the  day  had  to  be 
characterized  by  an  'Hmnatural"  occurrence.  Because  it  is  not  at  all 
normal  for  the  periods  of  the  day  to  be  characterized  by  w^neraia  that 
change  shspe  from  oval  to  rectangular,  this  occurrence  served  as  the 
unnatural  concept.  In  one  variation,  subjects  were  told  about  the  Hafo 
Indians  of  Western  Canada  \dio  use  the  names  stcbe,  kovit,  bren,  and  trofar 
to  describe  the  dedly  pattern  of  changes  in  the  position  of  the  sun.  m 
another  version  subjects  wei'a  told  about  the  planet  Ryrcn,  where  the  ■ 
periods  of  the  de^  are  called  dawn,  daytime,  dusk,  and  ni^ttime,  but  are 
marked  by  the  f ac±  that  a  certain  type  of  mineral  changes  shape  tram 
rectangular  to  oval  or  frcm  oval  to  rectangular,  m  the  final  variation, 
subjects  were  told  about  the  planet  ^rcn,  where  the  periods  of  the  day 
are  called  stobe,  kovit,  bren,  and  trofar,  and  can  be  distinguished  by  the 
fact  that  a  certidn  kind  of  mineral  changes  shE^  frcm  rectangular  to 
oval,  acxxsrding  to  a  systematic  pattern.  Eacdi  of  the  four  tasks  is 
summeud^ed  in  Table  35. 


Insert  Table  35  about  here 


The  main  ciifference  between  this  set  of  materials  and  the  earlier  one 
is  that  the  selectiorHask  items  were  not  drawn  on  cards;  instead,  they 
were  presented  on  a  ocxnputer  (CRT)  screen.  The  pic±ure  representations  of 
the  sun  and  the  horizon  were  drawn  using  five  hypens  and  an  "o",  with  the 
"o'*  appearing  either  0.5  cm  abcve  or  below  the  hyphens.  The  rec±angular 
and  oval  shapes  were  1  X  0.5  cm  and  they  were  drawn  using  a  set  of 
graphics  characters.  The  survey  was  carried  out  just  as  it  was  in  the 
fir^  epgperlment. 

Desi^.  The  same  cieslgn  was  used  in  this  esgperiment  as  in  the 
previous  one,  inclxidlng  the  cxuntodalancing  scheme  for  the  three 
variations  of  the  New  Haven  information. 

Acparatus.  In  contrast  to  the  first  experiment,  the  selection  task 
items  in  this  stucfy  were  presented  on  a  Televideo  950  CRT,  driven  by  a 
North  Star  Horizon  ocnputer.  Subjec:ts  slgneded  their  responses  by 
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pressing  one  of  three  keys  on  the  keyboard.  The  abilil^  tests  viere  agedn 
given  in  pe^jer-and-pencll  format. 

Prooedure.  Because  this  esqierlioent  was  run  1:^  a  oonputer,  there  were 
several  procedural  differences.  Cnee  a  subject  ooBopleted  the  practice 
trials,  the  es^erimenter  no  longer  played  an  active  role  in  the 
experiment.  Subjects  initiated  each  trial  by  pressing  the  space  bar. 
Following  a  1-s  delay,  a  problem  2^peared  on  the  center  of  the  CRT  screen 
and  an  internal  clook  was  started.  VSien  subjects  hit  one  of  the  three 
answer  keys,  the  clook  was  stopped,  the  solxition  time  was  recorded,  and 
the  problem  was  cleared  from  the  screen.  Subjects  were  allowed  to 
initiate  trials  at  their  own  rate.  Ihe  between  trials  was  not 
recorded.  Ihis  procedure  allowed  sulE^ects  the  opportunity  to  ask 
questions  when  they  were  not  clear  about  some  aspect  of  the  eaqjeriment. 
Si±fjects  were  also  notified  by  the  oonputer  when  they  made  three  errors  in 
a  row;  they  were  then  shown  the  correct  answer  to  these  problems  in  order 
to  ensure  that  they  understood  what  each  problem  regiilred.  Ihe  time 
needed  to  complete  this  part  of  the  eaq^eriment  varied  from  30  min  to  a 
full  hour.  All  other  aspects  of  the  prooedure  were  the  same. 

Results 

Basic  Statistics 

Main  experiment.  Basic  statistics  for  the  groips  defined  by  the  2X2 
design  are  provided  in  Table  36.  Soli±ion  times  are  again  based  on  108 
observations  per  subject,  averaged  over  the  20  subjects  in  each  groep. 

The  main  effect  of  concept  was  not  significant,  F(l,76)  -  2.48,  p  >  .05, 
and  the  medn  effect  of  language  also  was  not  significant,  F(l,76}  -  2.99, 
p  >  .05.  Ihe  concept  X  language  interaction  did  reach  significance, 
F(l,76)  =•  3.98,  p  <  .05,  replicating  the  major  finding  of  the  first 
experiiaent.  Hewever,  in  contrast  to  the  first  experiment,  the  error  data 
did  not  follow  the  same  pattern  as  the  reaction-time  data.  Ihe  concept 
main  effect  was  not  significant,  F(l,76)  -  .358,  p  >  .05,  but  there  was  a 
significant  main  effect  for  language,  F(l,76)  -  4.32,  p  <  .05,  and  the 
concept  X  language  Interaction  f^dled  to  reach  significance,  F(l,76)  » 

.99,  p  >  .05.  An  examination  of  the  mean  error  rates  shewed  that  the  two 
conditions  that  used  familiar  names  (groips  1  and  3)  had  the  highest  error 
rates.  It  appears  that  the  pattern  of  results  shewn  in  the  analysis  of 
variance  can  be  attributed  mainly  to  group  1,  which  had  a  different 
ordinal  ranking  for  errors  than  it  did  in  the  first  experiment. 


Insert  Table  36  about  here 


Oe^ite  the  anomaly  in  the  error  rates,  the  patterns  in  the  latency 
data  cannot  be  d1«atiisfiPd  as  being  a  result  of  a  ^eed-accuracy  tradeoff. 
Ihe  correlations  between  latencies  and  error  rates  oenputed  across  20 
subjects  were  positive:  .31,  .65,  .34,  and  .15  for  sets  1,  2,  3,  and  4, 
re^ectively.  Only  the  correlation  for  set  2  wms  significant,  p  <  .05, 
two-tadled.  Oorrelatlons  between  error  rates  and  latencies  across  the  36 
item  types  were  .65,  .62,  .63,  and  .55  for  sets  1,  2,  3,  and  4, 
respectively.  All  of  these  oorrelatlons  were  significant  at  the  .01  level 
or  better,  \;ising  two-tailed  tests. 

Beeaxise  the  dependent  variables  were  once  again  correlated,  a 
multivariate  analysis  of  variance  was  performed.  Ihis  analysis  shewed 
that  the  two  sain  effects,  as  well  as  the  interaction,  edl  fedled  to  reach 
significance.  Ihis  deviation  from  the  results  obtedned  in  the  first 
esperiment  can  probably  be  attributed  to  the  aberration  in  the  rank  order 
of  error  rates,  noted  above. 
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In  summary,  ed.thcui^  these  results  do  not  exactly  parallel  those 
obtedned  in  the  first  ea^seriment,  th^  do  support  the  basic  definition  of 
ncnentrenchment,  as  described  by  Mcxiel  4. 

Survey.  In  the  first  survey,  89%  of  the  re^nnses  were  in  accordance 
with  the  predicted  relationship;  that  is,  at  dawn,  the  sun  starts  out 
below  the  horizon  and  eventually  moves  above  the  horizon;  in  daytime,  the 
sun  is  above  the  horizon  during  the  entire  period;  at  dusk,  the  sun  starts 
out  above  the  horizon  and  eventually  moves  below  the  horizon;  and  in 
ni^icttime,  the  sun  is  always  below  the  horizon,  (1,  N  «  9)  «  161.7,  p 
<  .001.  Ihese  results  are  consistent  with  the  latency  data  in  showing 
that  this  system  describes  our  entrenched  way  of  thiridng.  Gnoe  agedn, 
edthoug^  subjects  agreed  that  it  was  not  nmannn  for  the  periods  of  the  day 
to  be  associated  with  changes  in  the  shape  of  certain  minerals,  67%  of  the 
respondents  agreed  that  one  pattern  of  changes  was  more  reasonable  than 
ary  other;  that  is,  at  dawn,  minerals  start  out  oval  and  eventually  become 
rectangular;  in  daytime,  th^  are  rectangular  throu^iout;  at  dusk,  they 
are  rectangular  at  the  beginning  and  oval  at  the  end;  and  in  ni^ttime, 
they  are  oval  throu^iout,  (1,  N  ®  15)  =  145.5,  p  <  .001.  Of  the  four 
counterbalanced  versions  of  set  3,  A,  B,  C,  and  D,  subjects  in  the  surv^ 
identified  3A  as  being  the  most  meaning^.  The  mean  latencies  for  the 
four  subsets  of  set  3  were  6.07,  7.06,  6.51,  and  8.04,  respectively.  An 
analysis  of  variarx:e  on  these  four  forms  indicated  that  they  were  not 
significantly  different,  F(3,16)  =  .51,  p  >  .05.  As  in  the  first 
eiperiment,  it  appears  that  subjects  in  the  survey  may  have  associated  the 
shapes  with  the  relative  li^tness  or  darkness  of  the  various  periods,  hut 
this  comparison  did  not  aiffect  performance  in  the  reasoning  task. 
cxianti-fcative  Tests  of  the  Information-Prooessina  Model 

Table  37  shews  the  results  from  applying  the  information-processing 
model  to  this  data  set.  Once  agadn,  these  data  are  based  only  on  items 
that  were  answered  correctly,  adthough  there  were  no  noteworthy 
differences  for  the  latencies  that  included  trials  with  errors. 


Insert  Table  37  about  here 


In  addition  to  these  group  statistics,  the  model  was  able  to  account 
for  individual  subjects'  latency  data,  with  values  for  each  of  the 
four  groups  were  .70,  .64,  .64,  and  .70,  respectively.  The  regression  F 
for  grexp  4  is  lower  than  it  is  for  the  other  groups,  although  it  is  still 
hi^ily  significant.  Also,  the  residuals  for  grexp  1  were  not 
significantly  correlated.  De^ite  these  differences,  the  various 
statistics  shewn  in  Table  39  indicate  that  the  proposed  model  also 
provides  a  very  good  fit  for  the  data  in  this  experiment. 

Parameter  Estimates  for  the  Information-Processing  Model 

The  parameter  estimates  are  presented  in  Table  38.  All  the  parameter 
estimates  were  significantly  different  from  zero  across  the  four  groups, 
shewing  the  generality  of  the  processes  being  measured.  Once  agsdn,  the 
CRS  parameter  was  large  only  in  set  3  and  not  in  set  2.  However,  in 
general  the  longest  estimates  are  not  consistently  found  in  sets  2  and  3, 
as  in  the  first  experiment,  and  the  five  estimates  do  not  follcw  a 
consistent  order  in  terms  of  their  relative  size.  These  results  may  stem 
hxn  the  fact  that  the  subject  reliabilities  were  lower  than  they  were  in 
the  first  experiment:  .82,  .73,  .72,  and  .77,  for  sets  1,  2,  3,  and  4, 
respectively,  compared  to  .84,  .85,  .86,  and  .86,  in  the  fir^ 
experiment.  Hence,  the  first  experiment  may  provide  more  interpretable 
information  regarding  the  various  processes  tliat  are  influenced  by 
nonentrenchment  because  these  parameter  estimates  are  based  on  more  stable 
data. 
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Insert  Table  38  about  bans 


Differences  Analvela 

Ibe  test  scores  were  again  ccndsined  to  form  ocnposite  scores  for 
inductive  and  deductive  reascning  ability.  CJorrelaticns  of  psycbonetric 
test  scores  with  the  mean  latencies  and  the  parameter  scores  are  shown  in 
Table  39. 


Insert  Table  39  about  here 


As  in  the  other  eispects  of  this  stxxfy,  the  correlations  are  similar  to 
those  in  the  first  experiment,  but  there  are  some  clear  differences.  The 
hi^xest  correlations  are  found  in  grcx^  2  and  3,  the  two  versions  using 
nonentrenched  concepts.  Althouc^  the  correlations  of  reaction  time  with 
the  psychonetric  test  scores  are  interpretable  in  terns  of  a  relationship 
between  intelligence  and  the  ability  to  reason  with  nonentrenched 
concepts,  they  do  not  corra^iond  to  the  results  of  the  first  eag)eriment. 
Perh2^  the  most  iaportant  regularity  to  be  found  in  these  experiments  is 
the  fact  that  set  3  had  the  longest  mean  reaction  time,  as  well  as  many  of 
the  hic^iest  correlations.  Ihese  results  suggest  that  using  familiar  names 
to  denote  xnnatuial  occurrences  may  be  the  more  interesting  form  of 
nonentrenchment;  this  was  the  most  difficult  conceptual  system  to  reason 
in  and  it  was  more  hi^ily  related  to  psychcmetric  measures  of  reasoning. 
Because  this  work  represents  an  early  attonpt  to  e9q>lore  ncnentrenchment, 
this  issue  may  bear  further  research. 

General  Discussion 

Ihe  oqperiments  described  in  this  report  were  carried  cut  to  e9^>and  on 
Sternberg's  (1982)  initial  iiivestigations  of  nonentrenchment.  Ihe  present 
work  examined  how  the  variables  of  lexical  familiarity  and  conceptual 
naturalness  function  in  making  a  concept  more  or  less  entrenched  in  our 
everyday  way  of  thinking.  Ihe  results  showed  that  these  variables  do  not 
have  independent  effects  on  reasoning.  As  described  by  Model  4,  the  locus 
of  nonentrenchment  lies  in  mapping  unnatural  occurrences  onto  familiar 
names  or  natural  occurrences  onto  x»vel  names.  Ihus,  information  that  is 
entrenched  is  not  sirply  familiar,  and  information  that  is  nonentraicfaed 
is  not  sinply  unfamiliar. 

Because  these  ea^aeriments  used  a  ^ecially  designed  reasoning  task,  it 
was  iaportant  to  assess  the  interral  and  external  validity  of  this  task. 

An  informatian-prooessing  model  for  the  reasoning  task  was  intemedly 
vedidated  by  showing  that  it  provides  a  good  account  of  latency  data  in 
two  experimoits,  and  was  externally  vali^ted  by  showing  that  parameter 
estimates  are  correlated  with  psychcnetrically  derived  reasoning  scares. 
The  model  edso  provides  some  eoq)lanatory  Information  concerning  the 
various  processes  that  might  be  eiffected  by  ncnentrenchment. 

This  research  i;sed  Nelson  Goodman's  (1955)  philosophical  ideas  as  a 
basis  for  e}q)lorlng  the  psychological  reality  and  nature  of 
ncnentrenchment.  Althcui^  the  enplrlcal  analysis  of  this  problem  was 
strai^xtforward,  the  interpretation  of  these  results  raises  several 
iaportant  issues  that  are  concerned  with  how  this  research  relates  to 
other  issues  in  cognitive  psychology. 

First,  our  analysis  of  the  entrenchment  constiaict  is  based  on  the 
finding  that  hi^ily  overleamed  names  or  concepts  can  produce  interference 
vhen  paired  with  unfamiliar  names  or  concepts.  Ihe  basic  form  of  this 
interference  is,  of  course,  not  new — it  is  a  variation  on  the  A-B,  A-C 
paradigm  frcn  associative  learning.  Ihe  nature  of  the  negative  transfer 
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effect  within  this  paradigm  has  been  analyzed  in  tezns  of  several 
different  frameworks,  such  as  proactive  interference  (Underwood,  1957) , 
organizational  theory  (Tulving,  1966) ,  and  list  discrimination  (Sternberg 
&  Bower,  1974) .  We  would  like  to  argue  that  nonentrencfaed  thinldng 
represents  a  particularly  interesting  form  of  negative  transfer.  Bower 
(1974)  pointed  out  that  nothing  in  interference  theory  restricts  its 
2^1icability  to  meaningless  materials  or  to  "factual  details."  He 
demonstrated  that  basic  interference  effects  can  be  produced  using  sirple 
prose  passages.  In  our  woidc,  subjects  ea^erlenced  interference  because 
they  were  trying  to  reason  with  new  kinds  of  information  that  were  only 
partially  meaningful  in  terms  of  their  prior  knowledge.  The  nonentrenched 
oonceptmd  systems  were  designed  to  be  "incorrect"  or  implausible  for  the 
subjects  who  participated  in  these  eaqaerdments;  however,  these  same 
oonoeptu£d.  systems  were  assumed  to  be  entrenched  for  other  subjects  frcm  a 
different  background.  (See  Kell,  1979,  for  a  discussion  of  implausibility 
and  the  related  phenomenon  of  ancnaly.)  The  distinction  between 
entrenched  and  nonentrenched  information  may  provide  a  different  way  to 
think  about  various  problems  in  learning  and  memory,  although  it  will 
remain  for  future  research  to  support  this  possibility. 

Second,  we  need  to  consider  why  a  conceptual  system  that  uses 
unfamiliar  names  to  denote  trnmatural  occurrences  is  no  more  difficult  to 
reason  with  than  an  entrenched  ooncRptual  system.  Althcu^  we  have 
explained  the  difficulty  associated  with  nonentrenched  thinking  in  terms 
of  negative  transfer  from  prior  knowledge,  it  is  possible  that  there  is 
also  a  facilitation  effect  that  is  operating  within  this  paradigm.  Bower 
(1974)  shewed  that  the  conceptual  macroetructure  of  a  text  can  be  a  source 
of  retroactive  facilitation,  vhether  the  underlying  details  of  the  text 
are  changed  or  kept  the  same.  However,  other  research  (Bower,  Blacdc,  & 
Turner,  1979,'  Experiment  5;  Klntsch,  Handel,  &  Kozminsky,  1977)  has  shown 
that  a  oono^Ttual  maczostructure  can  be  a  source  of  proactive  interference 
when  subjects  are  required  to  rempmber  jumbled  texts.  In  our  studies  of 
nonentrenchment,  we  used  four  ocnceptajal  systems  that  hzxi  different 
"surface”  structures  (contents)  generated  from  a  ociancn  "deep"  structure 
(conceptual  macrostriicture) .  Hence,  the  oenmnn  ocnceptiial  structure  may 
have  been  a  source  of  proactive  facilitation.  It  will  remain  for  future 
research  to  demonstrate  the  possibility  that  there  are  edso  forms  of 
nonentrenched  thinking  in  which  a  familiar  ooncqTtual  structure  can  be  a 
source  of  negative  transfer  and  proactive  interference. 

Third,  the  present  eaperiments  were  specifically  designed  to  stuty  the 
nature  of  nonentrenchment.  However,  they  isplicitly  dealt  with  another 
problemr-the  relationship  between  language  and  thoui^.  The 
language-thou^t  problem  deeds,  in  part,  with  the  following  issue:  Every 
language  must  be  flexible  enou^  to  express  the  diverse  experiences  of  a 
culture;  yet,  a  specific  language  eppears  to  shepe  or  bias  the  way  the 
menbers  of  a  culture  think  about  their  eaperienoes.  (See  dark  and  dark, 
1977,  for  a  sunmary  of  various  epproaches  to  this  problem.)  Our  research 
relates  to  this  problem  in  that  it  dsoaonstrates  a  situation  in  vhich  the 
effects  of  linguistic  and  conceptual  factors  are  not  ind^endent. 

Although  the  potential  relationship  between  these  two  lini^  of  work  is 
interesting,  it  is  important  to  lirdt  the  scope  of  this  observation. 
Nbnentrenchment  was  operationalized  \;ising  two  very  narrowly  defined 
variables.  lexical  familiarity  does  not  begin  to  capture  the  structural 
and  semantic  ocoplexity  of  language.  And  most  concepts  cannot  be 
described  using  only  two  dichotemous  states.  Hence,  the  results  reported 
here  should  be  viewed  as  providing  a  basis  for  further  research  on 
nonentrenchment  and  its  relationship  to  the  language-^thou^t  problem. 
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Fourth,  Goodman's  (1955)  ideas  were  originally  intended  to  provide 
philosophers  with  a  different  perspective  on  the  nature  of  inductive 
logic.  CXir  enpirical  analysis  of  ncnentrencfaaent  nay  therefore  have 
inplications  for  the  way  psychologists  have  studied  the  way  we  nake 
inikictive  judgments.  Much  of  the  psychological  research  in  this  area  has 
attenpted  to  describe  the  heuristics  and  biases  that  we  use  in  making 
inductive  judgments  (Kahneoan,  Slavic,  &  Tversky,  1983) .  Our  research  nay 
relate  to  this  literature  in  that  «hen  we  are  required  to  make  jvidgments 
under  unoert2dn  conditions,  we  are  essentially  reasoning  in  ncnentrenched 
conceptual  systans.  Ideas  about  base-rate  probabilities,  sanple  sizes, 
and  covariation,  for  exanple,  are  ncnentrenched;  our  intuitive  theories 
about  these  concepts  are  not  easily  mcTdified  and  they  interfere  with  our 
attempts  to  make  judgments  that  would  be  consistent  with  these  concepts. 
Hence,  it  may  be  \:iseful  to  e;g>lore  how  the  ocnstruct  of  ncnentrenchment 
relates  to  specific  heuristics  that  people  appear  to  use. 

Finally,  the  eapperimental  findings  on  ncnentrenchment  are  in 
aooordanoe  with  the  traditionally  held  belief  that  intelligenoe  is 
characterized  by  the  ability  to  acquire  new  knowledge.  We  hatve  drawn  our 
oonclxisions  by  assessing  individual  differences  in  the  ability  to  solve 
reasoning  prcblens,  according  to  a  ocnponential  framework  (Sternberg, 

1977) .  Raaheim  (1974)  has  formally  diioissed  many  of  these  basic  issues 
in  his  analysis  of  "pixiblen  situations."  According  to  his  theory, 
intelligence  is  most  important  to  problem  solving  when  the  problem 
situation  possesses  a  mlklium  amount  of  familiarity.  He  argued  that  if  a 
problem  is  very  familiar,  then  It  is  mastered  quite  easily;  if  it  is  very 
vnusual,  then  it  is  not  a  good  indicator  of  intelligenoe  because  a  chance, 
epqploratory  approach  often  provides  the  best  or  even  only  way  to  obtain  a 
solution.  The  present  data  on  nonentrenched  ocnoepts  axe  in  aooordanoe 
with  this  analysis,  in  addition  to  other  related  theories  put  forth  by 
Berg  and  Sternberg  (1985) ,  cattell  (1971) ,  Horn  (1968) ,  xaufinan  and 
Kaufinan  (1983) ,  Snow  (1981) ,  and  Steniberg  (1985) .  However,  it  is 
important  to  note  that  Schank  (1980)  has  argued  that  ocnventicnal  problem 
solving  is  an  inapprcprlate  way  to  understand  intelligenoe  because  it  is 
dependent  on  script-controlled  b^iavior.  Instead,  he  has  stated  that 
intelligenoe  is  characterized  by  ". .  .the  ability  to  make  generalizations 
frcm  ocmpletely  new  situations  that  are  useful  for  future  needs. ..." 
Schank  has  also  asserted  that  prior  cultural  informaticn  is  not  helpful  in 
\siderstanding  intelligenoe  because  ary  test  that  relies  on  this  exiting 
knowledge  will  simply  be  measuring  one's  prior  tradning.  However,  his 
analysis  is  not  sensitive  to  the  fact  that  cultural  knowledge  has  to  be 
acquired,  and  that  this  learning  occurs  over  a  number  of  years.  Our  work 
on  ncnentrenchment  indicates  that  Schank's  statement  is  too  eoctreme — It 
can  actually  be  relatively  easy  to  reason  in  entirely  new  conceptual 
systems.  As  noted  by  Sternberg  (1981) ,  intelligenoe  is,  at  least  in  part, 
the  ability  to  apply  existing  knowled^  structures  to  new  ccnoeptual 
systems. 

In  summary,  research  on  ncnentrenchment  is  potentially  relevant  to 
several  different  problems  within  cognitive  psychology.  Just  as 
Wittgenstein's  (1953)  work  influenced  the  way  psychologists  studied 
ocnoepts,  Goodman's  (1955)  ideas  about  entrenched  predicates  way  inflxience 
the  way  we  think  about  various  problems  in  learning  and  memory,  induction, 
and  ini^ligenoe. 


CNR  Oontract  NOOO148S0589  FinzuL  Report,  Page  51 


i4:pe3xlix  A 


Basic  bridge  rules 

Following  are  the  official  rules  of  bridge  quoted  fron  Borland 
Intematicnal's  handbook  of  Hirbo  Gaoeworks  (1985a}. 

Bridge  is  a  card  game  for  four  players,  two  pairs  of  partners.  The 
partners  sit  across  the  table  froon  cne  another,  with  an  opponent  on  each 
side.  For  convenience,  the  players  might  be  referred  to  as  "North", 
"East",  "South",  and  "West"  to  oorre^snd  with  their  geographical 
positions  around  the  table.  Ihus  North  and  South  are  partners.  East  and 
West  are  partners,  and  around  the  table  clododse  are  North,  East,  South, 
and  West. 

Each  player  is  dealt  13  cards.  The  object  of  the  game  is  for  one 
partnership  to  oontract  to  win  a  cert2dn  number  of  tricks,  \diile  the 
apposing  pair  attempts  to  prevent  this.  A  trick  consists  of  four  cards, 
one  played  by  each  player.  Since  each  player  has  13  cards,  there  are  13 
tricks  avedlable. 

Ihe  ple^  to  a  trick  is  as  follows:  one  p>layer  p)lays  any  card,  and  the 
oth^  three  players  each  play  a  card  in  turn,  going  clockwise  around  the 
table.  A  player  must  "follow  suit,"  that  is,  play  a  card  of  the  same  suit 
as  the  fir^  card  of  the  trick  if  she  is  able  to  do  so~if  not,  she  nay 
play  any  card.  The  winner  of  the  trick  is  the  p>layer  who  p>lays  the 
highest  card  of  the  suit  led  (ace  is  hi^,  deuce  is  low) ,  unless  there  is 
a  trump  suit  (a  special  suit  determined  by  the  bidding)  in  which  case  the 
hi^iest  trump  played  wins  the  trick.  Ihe  person  who  wins  a  trick  plays 
first  to  the  next  tzlck,  and  play  continues  until  all  13  tricks  have  been 
played. 

The  determination  of  the  trump  suit  and  which  pair  contracts  for  how 
many  tricks  is  done  ty  an  auction — the  hic^iest  bidder  gets  the  contract. 
The  rank  order  of  the  suits  is  clubs  (lowest) ,  diamonds,  hearts,  spades, 
and  notrunp.  The  smallest  number  of  tricks  one  can  oontract  for  is 
seven.  Six  is  added  to  the  bid,  and  that  is  the  number  of  tricks  which 
the  bidder  is  contracting  to  win.  If  two  bids  oontract  for  the  same  number 
of  tricks,  the  one  with  the  hic^ier  ranking  suit  is  the  hi^ier  bid,  but  a 
bid  for  more  tricks  edways  outranks  a  bid  for  fewer  tricks. 

The  auction  proceeds  as  follows:  starting  with  the  dealer,  and  going 
clockwise,  each  player  has  the  option  of  either  passing  or  making  a  bid 
hitler  than  the  last  bid.  m  ad^tion,  if  an  opponent  has  made  the  last 
bid  one  has  the  option  of  doubling,  and  if  an  opponent  has  doubled  one's 
bid,  of  redoubling.  These  doubles  and  redoubles  increase  both  the  rewards 
for  making  the  oontract  and  the  penalties  for  losing  the  contract.  The 
bidding  continues  until  there  are  three  consecutive  passes.  One  is 
permitted  to  make  a  legitimate  bid  even  if  one  has  passed  earlier  in  the 
auction. 
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The  primary  goal  in  the  bidding  is  to  choose  the  fined,  contract  such 
that  the  peu±nership's  best  suit  is  tnmps,  and  to  contract  for  game  if 
the  partnership  can  win  encu^  tricks.  Since  hi^  cards  win  tricks, 
eveduaticn  of  the  strength  of  a  hand  di^)ends  primarily  \Jipcn  the  hi^  card 
content.  The  evaluation  technique  used  by  almost  all  players  today  is  as 
follows: 

ace  a  4 

king  -  3 

queen  »  2 

jack  s*  1. 

There  are  a  total  of  40  hi^  card  points.  Players  usually  want  a 

hand  to  be  at  least  one  ]dng  above  average,  13  or  more  points,  to  make  the 
first  bid,  edLthoui^  this  requirement  can  be  relaxed  when  one  has  a  long 
suit  (5  or  6  cards) ,  since  long  suits  produce  extra  tricks  in  the  play. 

Vhen  the  bidding  has  concluded,  the  pedx  that  made  the  hi^iest  bid  is 
the  declaring  side,  and  the  player  of  that  pair  vho  first  named  the  suit 
of  the  final  omitract  is  the  declarer. 

The  play  proceeds  as  follows:  the  player  to  the  left  of  the  declarer 
leads  to  the  first  trick.  After  this  lead,  declarer's  partner  (called  the 
dumttY)  puts  his  or  her  cards  face  vp  on  the  table.  The  dunny  has  no 
further  say  in  the  proceedings — ^the  declarer  plays  both  dumny's  cards  and 
her  own.  Play  to  the  first  trick  continues  with  declarer  playing  a  card 
from  dunsiY/  then  the  player  ti.  declarer's  ri^t  playing  to  the  trick,  and 
finally  declarer  playing  the  trick.  The  winner  of  the  trick  leads  to  the 
next  trick,  and  play  continues  until  all  13  tricks  have  been  ccnpleted. 

If  declarer  has  won  at  least  the  number  of  tricks  contracted  for,  she  has 
made  the  contract;  if  not,  she  has  been  defeated. 

If  the  declaring  side  fulfills  their  contract,  they  score  points, 
determined  by  the  number  of  tricks  taken,  the  contract,  and  the  truirp 
suit.  If  the  contract  is  defeated,  the  defenders  score  points. 
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^pendix  B 

Skill  lew/el  Cateaorv  System 


Category 

Brief  description 

1 

Beginner 

no  esperience 

2 

Novice 

seme  experience,  little  ability. 

3 

Advanced  Novice 

esperienoe,  sene  ability, 
na^se  sene  conventions. 

4 

Post-Novice 

20f  points,  can  occasionally  place 
in  open  games. 

5 

Average 

ncnonal  ability  level,  can  play  a 
cohesive  system,  not  very  ejperienced. 

6 

Bad  life  master 

usually  misapplies  kncx/ledge,  but 
can  play  rote. 

7 

Above  average 

can  play  well,  has  sene  theory,  often 
fast-rising,  mistakes  due  to 
inesperience. 

8 

Average  life  master 

experienced,  can  oftmi  hold  their 
own  with  t^  players. 

9 

Strong 

usually  does  well  in  tournaments. 

10  Expert 

a  string  of  tournament  successes, 
maybe  plays  professionedly. 
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i^pendix  C 

Cover  Story  for  Game  Modlflcatlcai  1— Change  of  Card  and  Suit  Names 

I  hope  ycM  enjoyed  the  games  you  played  so  far.  not  things  are 

beocming  a  bit  more  ocnpllcated. 

Boagine  the  following  situation:  You  are  traveling  in  a  foreign 
country  and  are  not  able  to  vnderstand  a  word  of  the  language  people  are 
speaking.  One  evening,  you  are  invited  to  join  a  grocp  of  for^gners  in 
your  hotel  to  play  a  card  game.  Very  soon  you  realize  that  the  game 
people  are  trying  to  e)g)lain  to  you  is  exactly  the  game  of  bridge  you  know 
from  hcme-~anly  with  different  names  for  cards  and  suits. 

CARDS:  J30C  is  now  EEIB 

QUEEN  is  now  DILL 
KING  is  now  lORK 
ACE  is  now  R[jrz 

suns:  CLUB  is  now  KAMER 

OIMCMD  is  now  RAM3G 
HEART  is  now  BIREF 
SPADE  is  now  E^JIAR. 

Everything  else  appears  to  be  exactly  the  same  as  in  the  game  as  you 
kncv  it. 

Please  try  to  cope  with  that  situation  in  the  following  games  and 
play  as  you  did  during  the  previous  three  games.  There  is  no  need  to 
learn  the  new  e^qnessions  by  heart.  The  oonputer  will  prcnpt  you  with  the 
new  expressions  vhen  it  is  your  turn  to  make  a  move. 

If  you  want  to  see  the  new  expressions  again  please  hit  <A>. 

Hhen  you  are  ready  to  start  the  next  game  hit  SPACE. 
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^^pendix  0 

A  Ocoplete  Listing  of  the  36  Different 
Problems  for  E>^)erdment  1 


Item 

Answer 

lA 

Sunoer 

Sunner 

G 

IB 

Winter 

Winter 

B 

2A 

Sunner 

Winter 

I 

2B 

Winter 

Sunner 

I 

3A 

SuEsner 

Fall 

B 

3B 

Winter 

Spring 

G 

4A 

Sunner 

Spring 

I 

4B 

Winter 

Fall 

I 

5A 

Fall 

Summer 

G 

5B 

Spring 

Winter 

B 

6A 

EaU 

Winter 

I 

6B 

Spring 

Summer 

I 

7A 

Fall 

Fall 

B 

7B 

Spring 

Spring 

G 

8A 

Fall 

Spring 

I 

8B 

Spring 

Fall 

I 

9A 

G 

Sunner 

G 

9B 

B 

Winter 

B 

IQA 

G 

Winter 

I 

lOB 

B 

Summer 

I 

llA 

G 

Fall 

B 

IIB 

B 

Spring 

G 

U?.. 

G 

Spring 

I 

12B 

B 

Fall 

I 

13A 

Summer 

G 

Summer 

13B 

Winter 

B 

Winter 

14A 

Winter 

G 

Spring 

14B 

Surmer 

B 

RlLI 

15A 

Fall 

G 

Summer 

15B 

Spring 

B 

Winter 

16A 

Spring 

G 

Spring 

16B 

FaU 

B 

Fall 

17A 

G 

G 

Sunner 

17B 

B 

B 

Winter 

18A 

G 

B 

Fall 

18B 

B 

G 

Spring 

Note.  Die  items  for  EDqierlisent  2  can  be  generated  1:^  replacing 
•'summer, ”  "fall,"  "winter,"  and  "spring,"  with  the  names  "daytine," 
"dusk, "  nic^ittime, "  and  "dawn, "  and  by  substituting  "A"  (above  the 
horizon)  and  "B"  (below  the  hwizcn)  in  place  of  "G"  (green)  and  "B" 
(brown) . 
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Footnotes 


^Ihls  research  was  done  in  collaboration  with  Joyce  GaStel. 

^Hiis  research  was  done  in  collaboration  with  Joyce  Gastel. 

^Ihis  research  was  done  in  collaboration  with  Peter  Frensch. 

'^Ihis  research  was  done  in  collaboration  with  Sheldon  Tetewsky. 

%he  revised  model  \ised  five  rather  than  seven  parameters  to  predict 
latencies  on  18  distinct  items.  To  do  this,  a  scheme  was  devised  in  which 
three  of  the  parameters,  VWPl — "variable  state  word  in  position  1,"  WP — "number 
of  words  in  premise,"  and  WA. — "number  of  words  in  answer,"  were  consolidated 
into  one  parameter,  ALL— "assess  initial  information  in  the  first  premise. "  An 
entirely  new  parameter,  JUS — "justification,"  was  added.  This  parameter  dealt 
with  the  fact  that  for  "inoonsistent"  {sxblems  that  had  a  variable-state  word 
in  the  second  position,  subjects  vKxild  have  to  JUSTUY  that  they  used  the 
correct  variable-state  season  to  solve  the  problem.  Also,  the  non^rtrenched 
parameter  was  not  vised  because  in  the  present  es^ieriments,  nonentrenchment  was 
defined  by  the  variables  of  conceptual  unnatural  ness  and  lexical  unfamiliarity, 
within  the  analysis  of  variance  design. 

^One  possible  e>^lanaticn  for  the  various  differences  may  be  that  the 
second  experiment  allcwed  subjects  to  do  the  task  without  interacting  with  the 
e}q}erimenter.  Of  the  two  experiments,  the  second  one  had  higher  mean  error 
rates;  in  addition,  several  subjects  had  to  be  replaced,  due  to  excessive  error 
rates — this  problem  did  not  occur  in  the  first  ej^ieriment. 
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Table  1 

Representative  Verification  Items  frcn  thB  Experiment 


Keyed  Response 

Familiar  Gounterfactual 

Presi:ppo6ition  Presipposition 

1.  Familiar  presi:pposition: 

Trees  need  water. 

Ocunterfactual  presipposition: 

Trees  eat  people. 


a. 

Librarians  eat  map)le  trees. 

False 

False 

b. 

Cherry-picking  requires  great  bravery. 

False 

True 

c. 

Trees  are  carnivorous. 

False 

True 

d. 

Trees  have  branches. 

'nnae 

True 

e. 

ITees  are  harmless. 

True 

False 

f. 

lunberjacks  chcp  down  trees. 

True 

True 

2.  Familiar  presipposition: 

Cats  are  furry. 

Gounterfactual  presippositian: 

Cats  are  strongly  magnetized. 

a.  Cats  "stick"  to  refrigerators. 

b.  Cats  attract  pe^)ercli^. 

c.  Cats  have  sharp  claws. 

d.  Cats  eat  iron  filings. 

e.  Cats  like  eating  fi^. 

f .  Catnip  is  metallic. 

g.  Suspended  cats  tend  to  face  east-west. 


3.  Familiar  presupposition: 

Kites  fly  in  the  edx. 

Gounterfactual  presipposition: 

Kites  run  on  gasoline. 

a.  Kites  epoodt  exhaust. 

b.  Kites  need  fuel. 

c.  Kites  have  tails. 

d.  Kites  need  wind. 

e.  Kites  have  four  vheels. 

f.  Kites  are  faster  than  edrplanes. 

g.  Kites  can  e}p>lode  when  th^  crash. 

h.  Kites  are  sold  in  stores. 


False 

True 

False 

True 

True 

True 

'nme 

False 

False 

False 

False 

False 

False 

True 

True 

True 

False 

True 

False 

True 

True 

True 

False 

False 

'nnae 

True 

False 

False 

False 

False 

I 
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Table  2 

Basic  Statistics 


Response  Time 

Error  Rate 

Ncn-Novel  Presupposition 

Mean 

SID 

Mean 

sro 

True  Statements 

2.20 

.57 

.02 

.03 

False  Statements 

2.36 

.72 

.04 

.04 

Novel  Presupposition 

Tnie  Statements 

2.57 

.73 

.06  .05 

False  Statements 

2.91 

1.02 

.13  .09 

Note:  Response  time  is  measured  in  seconds 
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Table  3 

Analysis  of  Varianoe  for  Response  Tines 


Anova  MS 

I 

Presi^posltion  (Novel,  Non-novel) 

1 

852906 

80.70*** 

Subject  X  Presi;$:position 

49 

10369 

Change  in  Response 

1 

434 

.05 

Subject  X  Change 

49 

9465 

Answer  (True  or  False) 

1 

130572 

17.29*** 

Subject  X  Answer 

49 

7552 

Presc^ipoeition  X  Change 

1 

3688 

.92 

Subject  X  Presc^fxssltion  X  Change 

49 

4020 

Pcesippositlon  X  Answer 

1 

1189 

.30 

Subject  X  Presiffxxsitlon  X  Answer 

49 

2516 

Change  X  Answer 

1 

14346 

2.10 

Subject  X  Change  X  Answer 

49 

6816 

Presupposition  X  Change  X  Answer 

1 

38940 

25.73*** 

Subject  X  Presigposltion  X  Change  X 

Answer  49 

1514 

Note:  Re^xxise  time  was  adjusted  for  the  nunber  of  wozds  in  the  statement. 


***  p  <  .01 
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T^le  4 

Oorrelatians  of  Response  Tine  and  Error  Rate  with  i^ility  Scores 


Cue 

letter 

Sets 

CIMM 

Syllogisms 

Cattell 

Culture  Fair 

Crossing 

Out  As 

Noo-Novel 

RT 

-.46*** 

—.66*** 

-.32* 

-.04 

ER 

-.18 

.02 

.04 

-.15 

Novel 

FT 

-.52*** 

-.68*** 

-.32* 

-.02 

ER 

-.16 

-.24 

-.U 

-.15 

*  p  <  .05 
***  p  <  .001 


CMR  Oontxact  N000148^589  Finad  Report,  Page  65 


Table  5 

laxcncny  of  Induction  Iteana  in  Four  Ibases  of  Induction  Researdi 

Similarity  to  IQ-Test  Items 

IQ-Test-Like  Un-IQ-Test-Like 

Analogies 

Familiar  Classifications  Everyday  Inductions 

Series  Ocmpletions 


Novel  Analogies 

Unfamiliar  Novel  dassificatlons 

Novel  Series  Oonpletioana 


Conceptual  Projections 


Insist  Problems 
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Table  6 

Sanple  Stimaliaa  Task  Items 

ftnalogigs 


Familiar  Relevant 

Chalk  is  used  for  writing. 

INK  :  P^^PER  ::  CHAIK  :  (a)  W3RD,  *(b)  BIACKBCARD,  (C)  ERASER, 

(d)  CIA5SR0CI1 

Familiar  Irrelevant 
Beds  have  sheets. 

BUHDING  :  DCME  ::  BED  :  *(a)  CANOPY,  (b)  PniCW,  (c)  HOTEL,  (d)  ROCN 
Mrn/ial  Relevant: 

Ihe  hand  is  the  organ  of  hearing. 

EXE  :  BUNDING  ::  HAND  :  (a)  SMOCOH,  (b)  TOUCHING,  *(c)  DEAFENING, 

(d)  WORN 

(Uncued  condition:  DEAFENING  is  replaced  hy  NUMBING) 

Novel  Irrelevant 

Sparrcws  play  hopscotch. 

TROCTT  :  SCALX  ::  SPARROW  :  (a)  GRACEFUL,  *(b)  FEAIHERX,  (c)  SMALL, 

(d)  ENERGETIC 

Classifications 


Familiar  Relevant 

Many  fish  are  edible. 

lUNA,  SARDINE,  SOIE,  FLOUNDER,  (a)  BAIT,  (b)  TADFOIE,  (c)  SEAFOOD, 
*(d)  SAIMON 

Familiar  Irrelevant 

Soup  is  mainly  water. 

BOUHICN,  EGG  CROP,  BROOH,  MINESTRCNE,  (a)  SAIAD,  *(b)  CHrtCER, 

(C)  LIQUID/  (d)  CRACKERS 

Novel  Relevant 

Flowers  grew  underground. 

TULIP,  DAFFODIL,  ROSE,  DAISX,  (a)  DIRT,  (b)  PETAL,  *(c)  CNION, 

(d)  FLOWERPOT  _ 

(lAxued  condition:  CNICN  is  replaced  by  BUITERCUP) 

Novel  Irrelevant 

Water  boils  at  roan  tenperature. 

FOG,  STEAM,  VAPOR,  CLOUD,  (a)  lUDDIE,  (b)  ICE,  *(c)  MIST,  (d)  RAIN 
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Table  6  (continued) 

Series  Ocnpletions 

Fami  1  i  ar  Pal 

Some  people  are  heetvier  than  others. 

SKINNY  ;  SLIM  :  AVERAGE  :  PLUMP  :  (a)  HUNGRY,  (b)  IHIN,  *(c)  FAT, 
(d)  AIHIEnC 

Familiar  Irrelevant 

Feet  are  bigger  than  hands. 

mUMB  :  INDEX  FINGER  :  MIDOLE:  FINGER  :  RING  FINGER  :  (a)  V^RIST, 

(b)  TOE,  *(C)  PINKY,  (d)  HAND 

Novel  Relevant 

Furniture  is  eaten  at  the  end  of  a  meed. 

APPETIZER  :  SOUP  :  SALAD  :  MAIN  COURSE  :  *(a)  TABLE,  (b)  BREAD, 

(c)  MENU,  (d)  ESTTREB 

(Uhcued  condition:  TABLE  is  replaced  by  DESSERT) 

Novel  Irrelevant 

People  always  sle^  standing  tjp. 

ANAKE  :  FATIGUED  :  CRONSY  :  ASLEEP  :  (a)  ALERT,  (b)  TIRED, 

(C)  OCNH3SED,  *(d)  UNCONSCIOUS 


Note:  Ihe  correct  answer  is  preceded  by  an  asterisk  in  the  table. 
Subjects  were  instructed  to  answer  items  as  thou^  the  premise  were  true, 
assuming  nothing  else  out  of  the  ordinary  other  than  the  information  in 
the  premise. 
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Table  7 

Reliabilities  of  Response  Tines 


Overall 

Uhcued 

Anedogies 

Item 

.89 

.89 

.89 

Subject 

.89 

.89 

.89 

Classifications 


Item 

.87 

.87 

.88 

Subject 

.91 

.91 

.91 

Series 


Item 

.88 

.85 

.90 

Subject 

.92 

.92 

.92 

Nbte:  Only  correct  response  tines  viere  used 
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Table  8 

Basic  Statistics 
Raw  Scores 

Anedogies  Classifications  Series  Overall 


RT  ER 


Familiar 

Cued 

3.91 

2.4 

Relevant 

Uhcued 

3.61 

0.7 

Familiar 

Cued 

4.18 

3.5 

Irrelevant 

uncued 

3.52 

2.2 

Novel 

4.97 

9.1 

Relevant 

Uncued 

3.82 

3.7 

Novel 

Cued 

4.32 

6.9 

Irrelevant 

uncued 

3.23 

2.4 

Note: 

Response  times  are 

e^^ressed 

FT 

ER 

RT 

ER 

RT 

ER 

3.81 

2.6 

5.15 

5.4 

4.29 

3.5 

4.48 

6.1 

5.55 

11.5 

4.55 

6.1 

4.26 

4.1 

6.22 

5.7 

4.89 

4.4 

3.87 

2.2 

5.17 

4.3 

4.19 

2.9 

6.64 

23.5 

5.22 

5.6 

5.61 

12.7 

4.26 

6.9 

5.01 

4.8 

4.36 

5.1 

4.42 

8.1 

6.65 

13.0 

5.13 

9.3 

4.09 

3.0 

5.02 

8.1 

4.11 

4.5 

seconds,  error  rates  in  percentages. 


Response  times  are  for  correct  responses  only 
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Table  9 

Analysis  of  Variance 
Reaction  Time  (Ocarrect  Only) 


Factors 

m 

I 

Test  type 

2 

302.1 

106.42*** 

Test  type  X  Subject 

118 

2.8 

Cue 

1 

163.8 

54.97*** 

Cue  X  Subject 

59 

3.0 

Novelty 

1 

36.7 

53.84*** 

Novelty  X  Subject 

59 

0.7 

Relevanoe 

1 

5.0 

10.88** 

Relevanoe  x  Subject 

59 

0.5 

Test  X  Cue 

2 

1.2 

1.15 

Test  X  cue  x  Suln^ect 

118 

1.1 

Test  X  Novelty 

2 

18.5 

28.29*** 

Test  X  Novelty  x  Subject 

118 

0.7 

Test  X  Relevance 

2 

39.7 

51.10*** 

Test  X  Relevance  x  Subject 

118 

0.8 

cue  X  Novelty 

1 

78.7 

153.06*** 

cue  X  Novelty  x  Subject 

59 

0.5 

Cue  X  Relevance 

1 

12.4 

14.71*** 

cue  X  Relevanoe  x  Subject 

59 

0.8 

Novelty  X  Rele'/anoe 

1 

18.3 

25.50*** 

Novelty  X  Relevanoe  x  Subject 


59 


0.7 
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Table  9  (ocntinued) 


Factors 

Test  X  Novelty  x  Relevance 

2 

16.3 

Test  X  Novelty  x  Relevance  x  Subject 

118 

0.7 

Cue  X  Novelty  x  Relevance 

1 

28.9 

Cue  X  Novelty  x  Relevance  x  Subject 

59 

0.7 

Test  X  Cue  x  Relevance  x  Novelty 

2 

20.8 

I 

24.91*** 

42.44*** 

23.95*** 


Test  X  Cue  x  Relevance  x  Novell  x  Subject  118 


0.9 
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Table  9  (oontinued) 
Anzdysis  of  Varianoe 
Error  Rate 


Factor 

m 

I 

Test  TVP^ 

2 

.1760 

23.75*** 

last  I^pe  X  Subject 

118 

.0074 

Cue 

1 

.2871 

28.08*** 

Cue  X  Subject 

59 

.0102 

Novelty 

1 

.4899 

89.59*** 

Novelty  X  Subject 

59 

.0055 

Relevanoe 

1 

.0879 

16.69*** 

Relevance  x  Subject 

59 

.0053 

Test  X  cue 

2 

.1416 

10.40*** 

Test  X  Cue  X  Subject 

118 

.0068 

Test  X  Novelty 

2 

.0907 

19.95*** 

Test  X  Novelty  x  Subject 

U8 

.0045 

Test  X  Relevance 

2 

.1385 

18.09*** 

Test  X  Relevance  x  Subject 

118 

.0076 

cue  X  Novelty 

1 

.4109 

44.01*** 

Cue  X  Novelty  x  Subject 

59 

.0093 

cue  X  Relevance 

1 

.0046 

.88 

cue  X  Relevanoe  x  Subject 

59 

.0052 

Novelty  X  Relevance 

1 

.0068 

1.15 

Mbvelty  X  Relevanoe  x  Subject 


59 


0062 
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Table  9  ((xntinued) 


Factor 

I 

Test  X  Cue  x  Novelty 

2 

.0586 

8.62*** 

Test  X  cue  x  Ndvelty  x  Subject 

118 

.0068 

Test  X  Cue  X  Relevance 

2 

.0635 

6.70* ** 

Test  X  Cue  X  Relevance  x  Subject 

118 

.0095 

Test  X  Novelty  x  Relevance 

2 

.2327 

29.78*** 

Test  X  Novelty  x  Relevance  x  Subject 

118 

.0078 

Cue  X  Novelty  x  Relevance 

1 

.1121 

17.29*** 

Cue  X  Novel'ty  x  Relevance  x  Subject 

59 

.0064 

*  p  <  .05 

**  P  <  .01 

***  p  <  .001 
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T^le  10 

Difference  Scores,  Precued  -  Uncued, 
for  Reaction  Tine  of  Correct  Responses  (seconds) 
and  for  Error  Rate  (percent) 

Analogies  dassifications  Series 


RT 

ER 

RT 

ER 

RT 

ER 

Familiar  Relevant 

.30 

1.7 

-.67 

-3.5 

-.40 

-6.1 

Familiar  Irrelevant 

.66 

1.3 

.39 

1.9 

1.05 

1.4 

Novel  Relevant 

1.15 

5.4 

2.38 

16.6 

.21 

0.8 

Novel  Irrelevant 

1.09 

4.5 

.33 

5.1 

1.63 

4.9 
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Table  11 

Oorrelatians  of  Response  Times  and  Error  Rates 
With  .Ability  Tests 

OVERAIIi  UNCUED  CUED 


RT 

ER 

RT 

ER 

RT 

ER 

DAT  Verbal  Reasoning 

-.21 

-.58*** 

-.19 

-.45*** 

-.23 

-.57*** 

Cattell  Abstract  Reasoning 

-.38** 

-.06 

-.32* 

-.02 

-.41** 

-.08 

Insist  Problems 

-.24 

-.41*** 

-.20 

-.36** 

-.26* 

-.39** 

Crossing  Out  A's 

-.09 

-.06 

-.16 

-.10 

-.04 

.04 

Extended  Vocabuleuy 

-.07 

-.29* 

-.10 

-.26* 

-.04 

-.26* 

(Level  3) 

Reasoning  Factor 

-.36** 

-.46*** 

-.31* 

-.36** 

-.39** 

-.45*** 

Verbal/Peroeptual  Factor 

-.10 

-.14 

-.17 

-.10 

-.05 

-.15 

Nbte:  Only  correct  response  times  were  included. 
*  P  <  .05  **  p  <  .01  ***  p  <  .001 
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Table  12 

Correlations  among  Expertise-Related  Variables 


Age 

Exprts 

Estlvl 

MPtot 

MPlast 

Yrsplyd 

Age 

0.15 

0.17 

0.06 

0.01 

0.32 

Exprts 

•it  "it 

0.77 

_  _  -  ★  ★  llf 

0.74 

0.76*** 

_  _  _  *  ★  * 
0.75 

Estlvl 

0.53** 

•ir  ^  •it 

0.56 

0.75 

MPtot 

0.95*** 

0.44* 

MPlast 

_  .  _  *  ★ 

0.48 

Yrsplyd 

Abbreviations:  Exprts: 

Estlvl : 
MPtot ; 

MPlast ; 

Yrsplyd: 


Rating  on  expertise  scale 
Self-estimated  level  of  expertise 
Total  number  of  master  points 
accvimulated 

Number  of  master  points  accumulated 
in  past  2  years 

Number  of  years  bridge  played  prior 
to  testing 


*  p  <  .05 

**  E  <  .01 

***  P  <  .001 
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Table  13 

Comparison  of  Expertise  Groups 


Variable 

Nonexperts 

Experts 

Mean 

Std 

Mean 

Std 

Age 

43.06 

15.85 

47.00 

8 .19 

Exprts 

2.06 

0.57 

6.51 

1.85 

Estlvl 

4.09 

1.75 

7.21 

0 . 92 

MPtot 

5.02 

7.93 

716.94 

963.01 

• 

MPlast 

2.48 

3.27 

186.53 

203.07 

Yrsplyd 

2.54 

1.81 

21.88 

8.67 
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Table  14 

Spearman-Brown  Corrected  Split-Half  Reliabilities  of 

Dependent  Variables 


Play  Time 

Tricks  Won 

Games  Won 

Overall 

0.92 

0.73 

0.63 

Normal  games 

0.85 

0.55 

0.51 

Modified  games 

0.89 

0.41 

0.40 
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Table  15 

Correlations  Among  Dependent  Variables 


Play  Time  Tricks  Won  Games  Won 


Play  Time 

Total  -0.11*  -0.07 
Unchanged  games  -0 .13  -0 . 12 
Modified  games  -0.08  -0.01 

Tricks  Won 

Total  0.36*** 
Unchanged  games  0.35*** 
Modified  games  0.36*** 


*  E  <  .05 


***  E  <  .001 


Table  16 
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Note:  Means  and  Standard  deviations  of  play  times  (per  play)  are  expressed 
in  seconds . 


Table  17 
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Note:  Means  and  Standard  deviations  of  play  times  (per  play)  are  expressed 
in  seconds  . 
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Table  18 

Basic  Statistics  for  Unchanged  Bridge  Rules  (Block  1) 


Variable 

Nonexperts 

Experts 

Mean 

Std 

Mean 

Std 

Play  Time 

11.86 

3.01 

9.33 

2.35 

Tricks  Won 

6.99 

1.43 

6.20 

1.74 

Games  Won  (%) 

0.51 

0.17 

0.65 

0.27 

Note:  Means  and  Standard  deviations  of  play  times  (per 
response)  are  expressed  in  seconds. 
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Table  19 

Change  in  Dependent  Variables  due  to  Game  Modifications 

Percentages 


Expertise 

Display  Change 

Play  Time 

Tricks  Won 

Games  Won 

Nonexperts 

Name  change 

5.46 

-5.94 

8.33 

Rank-order  change 

4.94 

-3.16 

-20.83 

Lead  change 

-0.89 

1.69 

40.00 

Experts 

Name  change 

12.31 

22.70 

-1.39 

Rank-order  change 

6.53 

30.95 

-5.95 

Lead  change 

33.68 

10.63 

-16.67 

Note:  Means  of  changes  in  play  time,  number  of  tricks  won, 

and  number  of  games  won  are  expressed  in  percent. 
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Table  20 

Basic  Statistics  for  Block  3 


Variable 

Nonexperts 

Experts 

Mean 

Std 

Mean 

Std 

Play  Time 

12.20 

2.61 

9.17 

2.37 

Tricks  Won 

6.67 

1.29 

7.13 

1.73 

Games  Won  (%) 

0.39 

0.29 

0.51 

0.29 

Note:  Means  and  Standard  deviations  of  play  times  (per 
response)  are  expressed  in  seconds. 
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Table  21 

Bid  time  (opening  bids) 
Alpha  Reliabilities 


Bid  Time 


Item 

Subject 

Overall 

0.93 

0.93 

Normal  rules 

0.88 

0.93 

Modified  rules 

0.91 

0.90 
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Table  22 

Bid  Time  (opening  bids) 
Raw  Data 


Modification 

Block 

Nonexperts 

Experts 

Mean 

Std 

Mean 

Std 

Name  change 

1 

8.69 

4.57 

7.06 

2.78 

2 

17.25 

11.22 

18.61 

12.74 

3 

15.47 

11.32 

13.79 

9.40 

4 

9.78 

5.73 

8.21 

4.29 

Rank-order  change 

1 

9.52 

4.41 

7.06 

3.96 

2 

14.86 

9.43 

14.32 

11.20 

3 

15.26 

11.11 

10.14 

7.87 

4 

10.07 

5.84 

8.48 

7.75 

Lead  change 

1 

9.24 

6.38 

8.16 

7.74 

2 

12.75 

9.14 

13.78 

11.57 

3 

10.39 

6.91 

9.22 

8.14 

4 

9.92 

6.14 

7.66 

5.97 

Note:  Means  and  Standard 

deviations 

of  bid 

times  (per 

bid) 

are  expressed  in  seconds. 
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Table  23 
Encoding 

Alpha  Reliabilities 


RT 

Total 

Correct 

Overall 

Item 

0.93 

0.98 

0.94 

Subject 

0.83 

0.46 

0.69 

Normal  Display 

Item 

0.91 

0.95 

0.88 

Subject 

0.89 

0.41 

0.54 

Modified  Display 

Item 

0.86 

0.96 

0.91 

Subject 

0.62 

0.50 

0.76 

Abbreviations:  RT:  Reaction  time 

Total :  Total  number  of  cards  written  down 

Correct:  Number  of  correctly  identified  cards 
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Table  24 

Correlations  Among  Dependent  Variables 


RT 

Total 

Correct 

RT 

Total 

- 

-0.13*** 

-0.34*** 

Normal  Display 

- 

-0.10* 

-0.36*** 

Modified  Display 

- 

-0.17*** 

-0.33*** 

Total 

Total 

- 

0 .74*** 

Normal  Display 

- 

0.70*** 

Modified  Display 

- 

0.78*** 

Abbreviations:  RT: 

Total : 
Correct : 


Reaction  time 

Total  number  of  cards  written  down 
Number  of  correctly  identified  cards 


★  ★  ★ 


E  <  .  05 

E  <  .001 


Table  25 


Note:  Means  and  Standard  deviations  of  RT  are  expressed  in  seconds. 


Table  26 
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Note:  Means  and  Standard  deviations  of  RT  are  expressed  in  seconds. 
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Table  27 

Basic  Statistics  for  Normal  Display  (Blocks  1  &  4) 


Nonexperts  Experts 


Variable 

Mean 

Std 

Mean 

Std 

RT 

2.02 

0.41 

1.87 

0.39 

Total 

5.00 

0.83 

6.23 

1.90 

Correct 

4.60 

0.77 

5.31 

0.66 

Note;  Means  and  Standard  deviations  of  RT  are  expressed  in 
seconds . 


Abbreviations:  RT: 

Total : 
Correct : 


Reaction  time 

Total  number  of  cards  written  down 
Number  of  correctly  identified  cards 


1 
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Table  28 

Change  in  Dependent  Variables  due  to  Display  Changes 

Percentages 


Expertise 

Display  Change 

RT 

Total 

Correct 

Novices 

Name  change 

-3.64 

-2.93 

-1.97 

Rank-order  change 

9.18 

-1.95 

-6.81 

Control 

-9.77 

6.91 

14.41 

Experts 

Name  change 

-3.40 

-9.03 

-10.43 

Rank-order  change 

7.86 

-3.54 

-2.09 

Control 

-4.24 

0.82 

4.91 

Note:  Changes  in  RT,  Total,  and  Correct  are  expressed  in 

percent . 


Abbreviations:  RT:  Reaction  time 

Total : 


Correct : 


Total  number  of  cards  written  down 
Number  of  correctly  identified  cards 
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Table  29 


Ifodels  for  Ncsnentrencbed  Concepts 


LR3cical  familiarity 

Conceptual 

Model 

Familiar  Unfamiliar 

naturalness 

Natural 

ISmatural 

Nature^. 

Unnatural 

Natural 

Uhnatured. 

Natural 

unnatural 

Natural 

Uhnatuzal 
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I^le  30 

Definition  of  Concepts  for  Experiment  1 


Season 

name 

Beginning 

color 

Ending 

color 

Groip  1: 

Leaves  in  New  Haven — Natural  concepts  and  familiar  names 

Summer 

Green 

1 

Green 

Fall 

Green 

Brcwn 

Winter 

Brcwn 

Brcwn 

Spring 

Brcwn 

Green 

Grocp  2: 

Leaves  in  Latzania — ^Natural  conceits  aixi  vmfamiliar  names 

Soob 

Green 

Green 

Trit 

Green 

Brcwn 

Blen 

Brcwn 

Brown 

Mave 

Brcwn 

Green 

• 

Grojp  3: 

Rocks  on  Fyrcn--^Jhnatxiral  concepts  and  familiar  names 

Summer 

Blvie 

Blue 

Fall 

Blue 

Orange 

Winter 

Orange 

Orange 

Spring 

Orange 

Blvie 

Grocp  4: 

Rocks  on  i^rcn — Unnatured  oonc^jts  and  unfamiliar  names 

Soob 

Blue 

Blue 

Trit 

Blue 

Orange 

Blen 

Orange 

Orange 

Mave 

Orange 

Blue 

Note;  For  sets  2,  3,  and  4,  the  set  of  descriptions  listed  is  only 
one  of  four  possible  oounterbalanced  versions  that  were  used. 
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Table  31 

Basic  Statistics  for  Experiment  1 


Graqp 

Mean 

Standard  deviation 

ever  subjects 

Standard  deviation 

over  item  t^pes 

Response  latencies 

1 

3.83 

.92 

1.04 

2 

4.18 

1.33 

1.25 

3 

4.80 

1.56 

1.45 

4 

3.76 

1.02 

1.00 

Error  rates 

1 

.06 

.04 

.05 

2 

.08 

.04 

.06 

3 

.09 

.06 

.09 

4 

.05 

.03 

.04 

Note:  Mean  latencies  are  expressed  in  seconds.  Eacii  set  of  data  is 


based  v^xsn  24  subjects  and  36  item  types. 


• 
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Table  32 

Indices  of  Model  Fit  for  E:g»rdBient  1 

Grcx^ 

Index 

Definition  1 

2  3 

4 

Proportion  of  variance  .93 

accounted  for  in  latency 

data  by  the  model 

.93  .92 

.91 

r(xx) 

Item  reliability  .95 

.97  .97 

.97 

F  reg 

Regression  F  for  32.7* 

31.8*  28.8* 

23.5* 

• 

df 

Degrees  of  freedom  5.12 

for  reg 

5.12  5.12 

5.12 

r(Bes) 

Correlation  between  .80* 

residuals  of  observed 

from  predicted  values  for 

splits  on  raiv3ca  halves  of 

subjects,  corrected  by 

Spearman-BrcMn  formula 

.90*  .80* 

.74* 

FMSD 

Root-mean-square  deviation  .31 

of  observed  from  predicted 

latencies 

.37  .45 

.34 

• 

Note:  Model  fits  are  based  cn  18  data  points. 

1 
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Table  33 

Lateixry  Parameter  Estimates  for  Experiment  1 


Grotp 


Parameter 


name 

Parameter  description 

1 

2 

3 

4 

All 

Assess  initial  state  (  ,  ,1) 

1.70 

1.90 

1.18 

1.38 

CRS 

Change  in  representation 

0.83 

0.88 

1.22 

0.72 

system  (0-1) 

CPS 

Change  in  physical  state  (0-1) 

1.30 

1.64 

2.15 

1.44 

UCVS 

Une^qsected  lack  of  change  (0-1) 

0.85 

0.85 

1.36 

0.74 

JUS 

JVjstification  (0-1) 

1.34 

1.82 

2.27 

1.37 

Note:  All  parameter  estimates  were  significantly  different  fTcn  zero 
(p  <  .05). 
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Table  34 

Oorrelatians  Between  Projectlcn  Task  Scores  and 
Measured  iAbllities  for  Experiment  1 


Grxxp 


1 

2 

3 

4 

Induction 

Overall  response  latency 

-.63** 

-.14 

-.57** 

-.40** 

Overall  error  rate 

-.22 

-.02 

-.33 

-.10 

AH 

-.23 

-.11 

-.26 

-.31 

CPS 

-.46** 

.19 

-.49** 

-.26 

CPS 

-.53** 

-.24 

-.56** 

-.34 

tJCVS 

-.29 

.01 

-.56** 

-.40* 

JUS 

-.11 

-.12 

-.41* 

-.24 

Deduction 

Overall  re^xxise  latency 

-.50** 

.06 

-.33 

-.35 

Overall  error  rate 

-.14 

-.03 

-.31 

-.62** 

AU 

-.28 

-.20 

-.12 

-.40* 

CPS 

-.39* 

.07 

-.15 

-.28 

CPS 

-.30* 

-.24 

.-49** 

-.21 

ucvs 

-.17 

.19 

-.48** 

-.26 

JUS 

.18 

-.02 

-.27 

-.37* 

Insic^ 

Overall  response  latency 

-.22 

-.27 

-.53** 

-.25 

Mote:  All  tests  of  significance  are  cne-tailed. 
*  P  <  .05 

**  p  <  .01 
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Table  35 

Definition  of  Ocnoepts  for  Experiment  2 


Grov^ 

Period 

name 

Beginning  ehepe 
or  position 

Ending  shape 
or  position 

Dawn 

Below 

Above 

1:  Sun  in  New  Haven: 

Daytime 

Above 

Above 

Natural  concepts  and 

Dusk 

Above 

Belcw 

familiar  names 

Nl^ttime 

Below 

Below 

Trofar 

Below 

Above 

2:  Sun  In  Canada: 

Kovlt 

Above 

Above 

Natural  concepts  and 

Bren 

Above 

Belcw 

unfamiliar  names 

Stobe 

Below 

Below 

Dawn 

Rectangular 

Oval 

3:  Mineral  on  I^ron: 

Daytime 

Oval 

Oval 

Uhnatural  concepts  and 

Dusk 

Oval 

Rectangular 

familiar  names 

Nlc^ittime 

Rectangular 

Rectangular 

Itofar 

Rectangular 

Oval 

4:  Mineral  on  I^nxn: 

Kovlt 

OveQ 

Oval 

unnatural  concepts  and 

Bren 

Oval 

Rectangular 

unfamiliar  names 

Stobe 

Rectangular 

Rectangular 

Note:  For 


of  four  possible 


sets  2,  3,  and  4,  the  set  of  descriptions  listed  Is  only  cxie 
oounteidsedanced  versions  that  were  vised. 


I 
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Table  36 

Basic  Statistics  for  Eaq^eriment  2 


Grcuqp 

Mean 

Standard  deviation 

over  subjects 

Standard  deviation 

over  item  types 

Response  latencies 

1 

5.42 

1.44 

1.45 

2 

5.54 

1.91 

1.37 

3 

6.92 

2.56 

1.67 

4 

5.36 

1.36 

1.36 

Error  rates 

1 

.09 

.07 

.09 

2 

.08 

.07 

.07 

3 

.09 

.08 

.10 

4 

.05 

.06 

.06 

Note:  Mean  latencies  are  es^ressed  in  seconds.  Each  set  of  data  is 


based  upon  20  subjects  and  36  item  types. 
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Table  37 

Indloes  of  Ifadel  Fit  for  Experiment  2 


GXXXp 

Index 

Definition  1 

2 

3 

4 

r2 

Proportion  of  variance  .95 

accounted  for  in  latency 

data  by  the  model 

.91 

.86 

.93 

r(xx) 

Item  reliability  .96 

.97 

.97 

.95 

F  reg 

Regression  F  for  R^  41.7* 

23.6* 

15.2* 

29.8* 

df 

Degrees  of  freedom  5.12 

for  reg 

5.12 

5.12 

5.12 

r(Res) 

Oorrelation  between  .35 

residuals  of  observed 

from  predicted  values  for 

splits  on  randan  hedves  of 

subjects,  corrected 

Spearman-Brown  fonmila 

.67*  .71* 

.74* 

I9GD 

Root-mean-square  deviation  .39 

of  observed  &xm  predicted 

latencies 

.46 

.65 

.44 

Note: 


Model  fits  are  based  on  18  data  points. 


*  p  <  .01 
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Table  38 

latency  Parameter  Estimates  for  Experiment  2 


Parameter 

Grcxp 

name 

Parameter  description 

1 

2  3 

4 

AH 

Assess  initial  state  (  ,  ,1} 

2.31 

2.05 

1.97 

1.66 

CRS 

Change  in  r^>resentation 

1.33 

0.94 

1.81 

1.28 

system  (0-1) 

CPS 

Change  in  pt^ical  state  (0-1) 

1.60 

1.39 

1.49 

1.73 

vcvs 

Uhe:qpected  lack  of  change  (0-1) 

1.48 

1.18 

1.28 

1.47 

JUS 

Justification  (0-1) 

1.60 

1.95 

1.44 

2.03 

Note:  All  parameter  estimates  were  significantly  different  &an  zero 
(P  <  .05) . 
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Table  39 

Correlations  Between  Projection  Task  Soares  and 
Measured  Abilities  for  Es^eriment  2 


Groi^ 


12  3  4 


Induction 


Overedl  response  latency 

-.26 

-.52** 

-.52** 

-.31 

Overall  error  rate 

-.37 

-.52** 

-.35 

-.12 

AH 

-.17 

-.10 

.01 

-.42 

CRS 

-.05 

-.06 

-.31** 

-.06 

CPS 

-.35 

-.58** 

-.53** 

-.15 

ucvs 

-.19 

-.36 

-.07 

.24 

JUS 

-.17 

-.31 

-.54** 

-.28 

Deduction 


Overall  response  latency 

-.25 

-.36 

-.67** 

-.36 

Overall  error  rate 

—.26 

-.61** 

-.04 

.11 

AU 

-.19 

.03 

-.32 

-.55** 

CRS 

-.11 

-.20 

-.72** 

-.21 

CPS 

-.02 

-.20 

-.14 

.02 

ucvs 

.13 

.08 

.18 

.28 

JUS 

.12 

-.29 

-.37 

-.15* 

Insic^ 


Overall  response  latency 


-.18 


-.53**  -.32  -.13 


Note:  All  tests  of  significance  are  one-t2dled. 
*  B  <  .05 
**  B  <  *01 
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Figure  Caption 


Figure  1.  Model  of  infoonatlon  processing  for  precued  induction  problems. 
Figure  2.  Hypothetical  game  constellation 

Figure  3.  Nonesperts*  changes  in  bid  time  due  to  game  modifications 
(in  percent) 


Figu3:e  4.  E:perts'  changes  in  bid  time  due  to  game  modifications 
(in  percent) 


Figure  5.  A  strategy  model  for  solving  projection  problems.  Symbolic 
names  in  the  ipper  ri^t-'hand  comer  of  a  box  refer  to 
latency  parameters. 


READ  CUE 


I 


T 


Figure  1 


GAME  NO :  1 


North 


Declarer;  E/W  N/S 
0  0 

Contract:  2  v 


West 


East 

f  QT  2 
o  K  T  4  3 
o  T  8  6 
♦  73 


South 

♦  J  3 

V  A  8  7  5 
o  K  J  9  3  2 

♦  A  9 


North  East  South  West  South:  ? 

Card  ranks:  A,K,Q,J  J,9...2 
Suit  ranks:  Sp,  He,  Dt,Cl 


Figure  2 


Figure  5 


OMR  ocntract  N0001485KD589  Final  Report,  Page  105 


Section  2 
Riblications 


1985 


Berg,  C.  A. ,  &  Sternberg,  R.  J.  (1985) .  Response  to  novelty:  Oontinuity 
versus  discontinuity  in  the  developmental  course  of  intelligence. 

In  H.  Reese  (Ed.),  Advances  developnent  and  bdiavior  (Vol. 

19,  pp.  2-47) .  New  York:  Asadenic  Press. 

Berg,  C.  A. ,  &  Sternberg,  R.  J.  (1985) .  A  triarchic  theory  of 

intellectual  development  during  adulthood.  Develoomental  Review.  5, 
334-370. 

Davidson,  J.  E. ,  &  Steniberg,  R.  J.  (1985) .  Ocnpetenoe  and  perfomance 
in  intellectual  development.  In  E.  Neimaric,  R.  da  Lisi,  &  J.  H. 
Newman  (Eds.),  Moderators  of  copflsetenoe  (pp.  43-76).  Hillsdale, 

N7:  Erlbaum. 


Dawning,  C.  J.,  Sternberg,  R.  J.,  &  Roes,  B.  (1985).  Multicausal 

inference:  Evalxiation  of  evidence  in  causally  oonplex  situations. 
Jcnmal  of  ^^THmPfntal  psychology:  General.  239-263. 


Sternberg,  R.  J.  (1985).  All's  well  that  ends  well,  but  it's  a  sad  tale 
that  begins  at  the  end:  A  reply  to  Glaser.  American  Psychologist. 
IQ,  571-573. 


Sternberg,  R.  J.  (1985) .  implying  ocnponential  anzdysis  to  the  study  of 
in^vidual  differences  in  cognitive  (Skills.  In  C.  Reynolds  &  V. 
Willson  (Eds.),  Methodoloqiced.  and  statistical  advano^  in  the  study 
of  indlvjdtiai  differences.  New  York:  Plenum. 


Sternberg,  R.  J.  (1985).  Beyond  IQ;  A  trlarchic  theory  of  human 
intellloenoe.  New  York:  Cambridge  Uhiversity  Press. 

Sternberg,  R.  J.  (1985).  Ihe  black-vdiite  differences  and  Spearman's  g: 
Old  wine  in  new  bottles  that  still  doesn't  taste  good.  Bdiavloral 
and  Brain  Sciences.  Q,  244. 

Sternberg,  R.  J.  (1985) .  Cognitive  epproaches  to  intelligence.  In  B. 
Wolran  (Ed. ) ,  tfandhrov  of  intelligence.  New  York:  Wiley. 

Sternberg,  R.  J.  (1985) .  CcDponential  analysis:  A  recipe,  m  D.  K. 
Detterman  (Ed.),  current  topics  in  human  intelligence  (Vol.  1,  pp. 
179-201).  Norwood,  NT:  Ablex. 

Sternberg,  R.  J.  (1985) .  Controlled  versus  autcmatic  processing:  A 
r^ly  to  Fodor.  Behavioral  and  Brain  Sciences.  Q,  32-33. 

Sternberg,  R.  J.  (1985) .  Critical  thinking:  Its  nature,  measurement, 
and  improvement.  In  F.  R.  Link  (Ed.),  Essays  on  the  intellect  (pp. 
45-65) .  Alexandria,  VA:  Association  for  Svpervislan  and  Curriculum 
Development. 


CNR  Contract  N000148K«)589  Flral  Report,  Page  106 


Sternberg,  R.  J.  (1985) .  For  the  best  on  hew  to  test:  Review  of 
Essaitlals  of  tjsvcholoaical  testing  (4th  ed.)  /  by  Lee  J.  Cronbach. 
Contemporary  Psychology.  3fi,  377-378. 


Sternberg,  R.  J.  (1985) .  HLrnian  intelligence:  Ihe  model  is  the  message. 
Science.  230.  1111-1118. 


Sternberg,  R.  J. 
and  wisdon. 
607-627. 


(1985) .  Implicit  theories  of  intelligence,  creativity, 
■Tniimal  of  Personal  tv  and  Social  Psychology.  49, 


Sternberg,  R.  J.  (1985) .  Instrumental  and  ocnponential  approaches  to  the 
training  of  intelligence.  In  S.  Chipnan,  J.  Segad,  &  R.  Glaser 
(Eds.),  Ihinklng  and  leaminty  sicills;  Current  research  and  open 
(Vol.  2,  pp.  215-243).  Hillsdale,  NJ:  Erlbam. 

Sternberg,  R.  J.  (1985) .  Pretty  dose  to  idead,  anyway:  A  review  of  Ihe 
^ rfAa T  pTxtolem  solver.  HUman  Intelligence  Newsletter.  6,  Spring,  10. 

Sternberg,  R.  J.  (1985) .  Review  of  Heichenbaum,  Borland,  Grusen,  and 
Cameron's  "Metacognitive  assessment."  m  S.  YUssen  (Ed.),  The  growth 
of  reflection  in  t-he  child  (pp.  31-35) .  New  York:  Academic  Press. 

Sternberg,  R.  J.  (1985) .  Review  of  NOnverbad  Test  of  Cognitive  Skills. 

In  J.  V.  Mitchell  (Ed.),  The  ninth  inental  measurements  yearbook. 
Lincoln,  ME:  university  of  N^oraska  Press. 

Sternberg,  R.  J.  (1985) .  Review  of  Test  of  Cognitive  Skills.  In  J.  V. 
Mitchell  (Ed.),  The  ninth  measurements  yMThook.  Lincoln,  NE: 

university  of  Nebraska  Press. 

Sternberg,  R.  J.  (1985) .  Tacit  agreements  between  authors  and  editors. 
Behavioral  and  Brain  Sciences.  5,  746-747. 

Sternberg,  R.  J.  (1985) .  Teaching  criticad  thinJeing,  Part  1:  Are  we 
making  criticad  mistokes?  Hii  Delta  Karpan.  §[7,  194-198. 

Sternberg,  R.  J.  (1985).  Teaching  critical  thinking.  Fart  2:  Possible 
solutions.  Ehi  Delta  Karoan.  67,  277-280. 

Sternberg,  R.  J. ,  &  Baron,  J.  B.  (1985) .  A  statewide  approach  to 
measuring  critical  thinking  skills.  Educational  43, 

40-43. 


Sternberg,  R.  J. ,  &  Caruso,  D.  (1985) .  Practical  modes  of  knowing.  In 
E.  Eisner  (Ed. ) ,  iMmina  the  wavs  of  knowing  (pp.  133-158) .  Chicago, 
IL:  university  of  Chicago  Press. 


Sternberg,  R.  J. ,  &  Davidson,  J.  E.  (1985) .  Cognitive  development  in  the 
gifted  and  talented.  In  F.  D.  Horowitz  &  M.  O'Brien  (Eds.),  The 
gifted  and  talented;  A  developmental  perspective.  Washington,  DC: 
American  Psychologicad  Association. 


CNR  Oontract  N0001485K0589  Final  Report,  Rage  107 


Sternberg,  R.  J. ,  &  McStanara,  T.  P.  (1985) .  Ihe  representation  and 

processing  of  infomation  in  reeaHdme  verbal  cxanprE^iension.  In  S.  E. 

Entoretson  (Ed. ) ,  Test  desicm;  Oontributlons  from  osvcholoav. 
education,  and  psvchcpietrics  (pp.  21-43) .  New  York:  Academic  Press. 

Sternberg,  R.  J.,  &  Smith,  C.  (1985).  Social  intelligence  and  decoding 
skills  in  nonverbal  communication.  Social  Oocmitlon.  168-192. 

Steniberg,  R.  J. ,  &  Spear,  L.  C.  (1985) .  A  trlarchic  theory  of  mental 

retardation.  In  N.  Ellis  &  N.  Bray  (Eds.),  Intematianal  rwjew  of 

research  in  mental  retardation  (Vol.  13,  pp.  301-326).  New  York: 
Academic  Press. 

Wagner,  R.  K. ,  &  Sternberg,  R.  J.  (1985) .  Practical  intelligence  in 
real-world  pursuits:  The  role  of  tacit  knowledge.  Journal  of 
Personal  Itv  and  Social  Psveholoctv.  4§,  436-458. 


1986 

Davidson,  J.  E. ,  &  Sternberg,  R.  J.  (1986) .  Nhat  is  insic^t? 

Educational  Hori 7nnfl  ^  64,  177-179. 

Dillon,  R.  F. ,  &  Sternberg,  R.  J.  (Eds.)  (1986).  Oocmitlon  and 
instruction.  Orlando,  FL:  Academic  Press. 

Ifeunr,  D.  B. ,  &  Sternberg,  R.  J.  (1986) .  Analogical  reasoning  vrith  novel 
concepts:  Differential  attention  of  intellectually  gifted  and 
nongifted  children  to  relevant  and  irrelevant  novel  stimali. 

Oocmitive  Develoanent.  1,  53-72. 

Spear,  L.  C. ,  &  Sternberg,  R.  J.  (1986) .  Oognltive  assessment  with 
disabled  readers.  Special  Ser<d^oes  in  the  Schools.  1985-1986,  2 
(2/3),  71-84. 

Sternberg,  R.  J.  (Ed.)  (1986).  Advances  in  the  psychology  of  human 
intelligence  (Vol.  3) .  Hillsdale,  NJ:  Erlbaum. 

Sternberg,  R.  J.  (1986) .  Alternatives  to  the  triarchic  theory  of 
intelligence.  Behavioral  and  Brain  Sciences.  2,  581-582. 

Sternberg,  R.  J.  (1986) .  Cognition  and  instruction:  Why  the  marriage 
sometimes  ends  in  divorce.  In  R.  F.  Dillon  &  R.  J.  Sternberg  (Eds.) , 
Cognition  and  instruction  (pp.  375-382).  Orlando,  FL:  Acad^c 
Press. 


Sternberg,  R.  J.  (1986) .  A  framework  for  understanding  conceptions  of 
intelligence.  In  R.  J.  Sternberg  &  D.  K.  Detterman  (Eds.),  What  is 
intelligence?  Contemporary  viewroints  on  its  nature  and  definition 
(pp.  3-15) .  Norwood,  NJ:  Ablex. 


Sternberg,  R.  J.  (1986) .  The  future  of  intelligence  testing. 
Educational  Measurement;  Issues  and  Practice.  2(3),  19-22. 


Sternberg,  R.  J.  (1986) .  GENECES:  A  framework  for  Intellectual 
abilities  and  theories  of  them.  Intelligence.  IQ,  239-250. 


CNR  Contract  N0001485K0589  Fined  Report,  Page  108 


Sternberg,  R.  J.  (1986) .  Haste  makes  waste  versus  a  stitch  in  time?  A 
reply  to  Vemcn,  Nador,  and  Kantor.  Intelligence.  iQ,  265-270. 

Sternberg,  R.  J,  (1986).  Hew  to  prepare  for  the  Miller  Analogies  Test 
(4th  ed.).  Woodbury,  NY:  Barron's  Educational  Series,  Inc. 

Sternberg,  R.  J.  (1986) .  Identifying  the  gifted  through  IQ:  Khy  a 
little  bit  of  knowledge  is  a  dangaxus  thing.  Roeper  Review.  8, 
143-147. 


Sternberg,  R.  J.  (1986) .  In  defense  of  'critical  thinking'  programs. 
Education  Week.  6(6),  19. 

Sternberg,  R.  J.  (1986) .  Inside  intelligence.  AmaTHran  scient-isti.  74, 
137-143. 

Sternberg,  R.  J.  (1986).  Instructor's  ^^rTroanv  mtellicrence 

acpliedi  Uhttersfcarriincf  iryrreasira  veur  intellectual  aHlls.  San 
Diego,  CA:  Haroourt,  Brace,  Jovanovich. 

Sternberg,  R.  J.  (1986).  mteiiloenoe  acolied:  and 

increasing  veur  intellectual  s)cma.  san  Diego,  CA:  Haroourt,  Brace, 
Jovanovidi. 


Sternberg,  R.  J.  (1986) .  Intelligence  is  mental  self-government.  In  R. 
J.  Sternberg  &  D.  K.  Datterman  (Eds.),  What  is  intelligence? 
Oantewporarv  viewpoints  on  its  and  ejafinition  (pp.  141-148). 

Nbrwood,  NT:  Ablex. 


Sternberg,  R.  J.  (1986) .  Intelllgenoe,  wisdon,  and  creativity:  Three  is 
better  than  one.  Educational  Psychologist .  21,  175-190. 


Sternberg,  R.  J.  (1986) .  Introduction:  The  nature  and  scope  of 

practical  intelligenoe.  In  R.  J.  Sternberg  &  R.  K.  Wagner  (Eds.), 
Practical  inteUicrence:  Nature  and  origins  of  oonoxtenoe  in  the 
everyday  world  (pp.  1-10) .  New  York:  Cambridge  university  Press. 


Sternberg,  R.  J.  (1986) .  A  perspective  on  the  projects  for  Human 
intelligence:  Persrectiv^  and  prospects,  ty  R.  Kail  &  J.  W. 
PellegrinD.  Oontemporarv  Psychology.  31,  18-19. 


Sternberg,  R.  J.  (1986) .  Time  ce^sules  are  not  a  paiaoea;  A  r^ly  to 
Vernon.  Intelligence.  IQ,  277-279. 


Sternberg,  R.  J.  (1986) .  Toward  a  unified  thecty  of  human  reasoning. 
Intelligence.  IQ,  281-315. 


Stemberg,  R.  J.  (1986) .  A  trlarohic  theory  of  human  intelligenoe.  In 
S.  E.  Newstead,  S.  H.  Irvine,  &  P.  L.  Dann  (Eds.),  Human  assessment: 
cognition  and  motivaticn  (pp.  43-47) .  Dordrecht,  The  Netherlands: 
Martinus  Nijhoff . 


Stemberg,  R.  J.  (1986) .  A  trlarohic  theory  of  intellectued  giftedness. 
In  R.  J.  Sternberg  &  J.  E.  Davidson  (Eds.),  Conceptions  of  gjfharinogg 
(pp.  223-243) .  New  York:  Canbrldge  Uhiversity  Press. 


ONR  Ccntract  N0001485K0589  Final  Report,  Page  109 


Sternberg,  R.  J.  (1986) .  Mhat  is  adaptive?  Bdiavioral  and  Brain 
Scien^.  9,  207-208. 

Sternberg,  R.  J.  (1986) .  What  wculd  better  intelligenoe  tests  look 

like?  In  Measures  in  the  oolleoe  admissions  orooess;  A  OoUeoe  Board 
Oolloouium  (pp.  146-150) .  Itew  York:  College  Board. 

Sternberg,  R.  J. ,  &  Berg,  C.  A.  (1986).  Quantitative  integration: 
Definitions  of  intelligenoe:  A  ocnparison  of  the  1921  and  1986 
synposia.  m  R.  J.  Sternberg  &  D.  K.  Detterman  (Eds.),  What  is 
intelligence?  Contenip<^ry  viewpoints  on  its  na-hnna  and  definition 
(pp.  155-162) .  Norwood,  NT:  iUalex. 

Sternberg,  R.  J. ,  &  Bhana,  K.  (1986) .  Synthesis  of  research  on  the 

effec^veness  of  intellectual  skills  programs:  Snake-oil  reonedies  or 
miracle  cures?  Educational  TAaAgshlp.  ^{2),  60-67. 

Sternberg,  R.  J. ,  &  Davidson,  J.  E.  (Eds.)  (1986).  Conceptions  of 
giftedness.  New  Yoidc:  Cambridge  university  Press. 

Sternberg,  R.  J. ,  &  Davixason,  J.  E.  (1986) .  Conceptions  of  giftedness: 

A  map  of  the  terrain.  In  R.  J.  Sternberg  &  J.  E.  DavlcSson  (Eds.) , 
conceptions  of  giftedness  (pp.  3-18) .  New  York:  Cambridge  university 
Press. 

Sternberg,  R.  J.,  &  Detterman  D.  K.  (Eds.)  (1986).  What  is 

intelligence?  Contemporary  viewpoints  an  its  nature  and  definition. 
Norwocxi,  NT:  Ablex. 

Sternberg,  R.  J.,  &  Suben,  J.  (1986).  The  socialization  of 

intelligence.  In  M.  Perlmutter  (Ed. ) ,  Perspectives  an  jTTt»nftr!»-ual 
ctevelcproent  fVol.  19):  Minnesota  symposia  cnaid  psychology  (pp. 
201-235).  Hillsdale,  NT:  Erlbaum. 

Sternberg,  R.  J. ,  &  f^tgner,  R.  K.  (Eds.)  (1986).  Practical 

intelligenoe:  Nature  and  origins  of  cxnipetence  in  the  everyday 
world.  New  York:  Cambridge  university  Press. 

Tetewsty,  s.  J. ,  &  Sternberg,  R.  J.  (1986) .  Conceptual  and  lexicxd 
determinants  of  nonentrenched  thinking.  .rauTDai  of  Memory  and 
Targfiiarp.  25,  202-225. 

Wagner,  R.  K. ,  &  Sternberg,  R.  J.  (1986) .  Tacit  knowledge  and 

intelligenoe  in  the  everyday  world.  In  R.  J.  Sternberg  &  R.  K.  Wagner 
(Eds.),  Practical  intelligenoe:  Nature  and  origins  of  cxnpetenoe  in 
the  everyday  world  (pp.  51-83) .  New  York:  Cambridge  university 
Press. 


mi 


Baron,  J.  B.,  &  Sternberg,  R.  J.  (Eds.)  (1987).  Teaching  thinking 
skills;  Theory  and  practice.  New  York:  Freeman. 


Kaye,  D.  B. ,  Sternberg,  R.  J. ,  &  Fonseca,  L.  (1987) .  Verbal 
ocnprehension:  The  lexical  decccpositlon  strategy  to  define 
unfamiliar  words.  Intelligence.  11,  1-20. 


9 


CMR  Ocsntract  N0001485K0589  Final  Bepart,  Ifetge  110 


Kolligian,  J. ,  Jr. ,  &  Stamberg,  R.  J.  (1987) .  Another  look  at 

inteUigenoe  and  learning  disabilities:  A  reply  to  Reynolds'  'rep* . 
Jcurnai  of  Leamino  Disabilities.  325-326. 

KoUigian,  J.,  JT.^  &  Steniberg/  R.  J.  (1987).  mtelligenoe,  infomatian 
processing,  and  ^lecific  learning  disabilities:  A  trlarchic 
synthesis.  Journal  TAam-tnqr  Disabilities.  2S»  8-17. 

Marr,  D.  B. ,  &  Sternberg,  R.  J.  (1987) .  Ihe  role  of  mental  speed  in 
intelligence:  A  triauxhlc  per^sective.  In  P.  A.  Vemcn  (Ed. ) ,  Speed 
of  infonnaticn  processing  and  intelligence  (pp.  271-294).  Norwood, 

NT:  Ablex. 


Sincoff ,  J. ,  &  Sternberg,  R.  J.  (1987) .  IWo  faces  of  verbal  ability. 
UMligmCg/  11/  263-276. 


Spear,  L.  C. ,  &  Sternberg,  R.  J.  (1987).  An  informaticn-processing 
frameworic  for  inderst2mding  reding  disabilities.  In  S  Oed  (Ed. ) , 

social,  and  noircpevcfaolocrlcal  aspects  of 
(Vol.  2,  pp.  3-31).  Hillsdale,  N7:  Erlbaim. 


Spear,  L.  C. ,  &  Sternberg,  R.  J.  (1987) .  Teaching  styles:  Stsff 
develcpm^  for  teaching  thinking.  Journal  of  Staff  Develconent. 
a (3),  35-39. 

Sternberg,  R.  J.  (1987) .  Behavior  genetics  moves  beyond  percentages— At 
last.  B^iavioral  ar/l  Brain  Sciences.  IQ,  40. 


Sternberg,  R.  J.  (1987) .  Ocping  with  novelty  aivi  human  intelligence.  In 
P.  Moixis  (Ed. ) ,  Modelling  cognition  (pp.  57-91) .  Londcn:  Wiley. 


Sternberg,  R.  J.  (1987) .  A  day  at  develc|inented  downs:  Sportscast  #2- 
Neo-Piagetian  theories  of  cognitive  development,  internatinnai 
Joamal  of  Psycholoav.  32(5/6),  507-529. 

Sternberg,  R.  J.  (1987) .  Difficulties  in  conparing  intelligence  across 
species.  Behavlorai  and  Brain  Sciences.  iQ(4),  679-680. 


Sternberg,  R.  J.  (1987) .  Down  with  paper  clipsl  A  review  of  V. 
John-Steiner,  NOt^xxte  of  the  mind:  Explorations  of  thinking. 
Oonteniporarv  Psychology.  33,  339-340. 

Sternberg,  R.  J.  (1987) .  Five  ways  to  think  about  thiiiking  skills. 
Instructor.  Special  Issue,  Fall,  32-33. 


Sternberg,  R.  J.  (1987) .  Gee,  there's  more  than  g:  A  critique  of  Arthur 
Jensen's  views  on  intelligence.  In  S.  Modgil  &  C.  Modgll  (Eds.), 
Arthur  Jensen:  Consensus  and  oontroversv  (pp.  237-249) .  Barocnbe, 
ihgland:  Falmer  Press. 

Sternberg,  R.  J.  (1987) .  Inplicit  theories:  An  edtematlve  to  modeling 
oognlticn  and  its  development.  In  J.  Bisanz,  C.  Brainerd,  &  R.  Kail 
(Eds.),  Formal  methods  in  developmental  psychology:  Progress  in 
cognitive  development  research  (pp.  155-192) .  New  York: 
Springer-Verlag. 


J 


CNR  Gcntract  N0001485K0589  Filial  Report,  Page  111 


Sternberg,  R.  J.  (1987).  Inbelllgenoe.  m  R.  L.  Gregory  (Ed.),  The 
Oxford  oopoanion  to  the  mind  (pp.  375-379).  Oxford,  England:  Oxford 
university  Press. 

Sternberg,  R.  J.  (1987) .  Intelllgenoe  and  cognitive  style.  In  R.  E. 

Sncw  &  M.  J.  Farr  (Eds.),  instruction.  Vol. 

ll  Oonative  and  affective  tarooess  analyses  (pp.  77-97).  Hillsdale, 
NT:  Erlbaum. 

Sternberg,  R.  J.  (1987) .  Most  vocabulary  is  learned  frcn  ocntext.  m  M. 
G.  MdRweon  &  M.  E.  Curtis  (Eds.),  The  nature  of  vocabulary  aoauisltlon 
(pp.  89-105).  Hillsdale,  NT:  Erlbaum. 

Sternberg,  R.  J.  (1987) .  The  psychology  of  verbal  ocnprdiensicn.  In  R. 
Glaser  (Ed.),  Advances  in  instaructlonal  psychology  (Vol.  3,  pp. 
97-151).  Hillsdale,  NT:  Erlbaum. 

Sternberg,  R.  J.  (1987) .  Questlcning  and  intelllgenoe.  Ouestlcnino 
Sjfidbaogs,  I,  11-14. 

Sternberg,  R.  J.  (1987) .  Questlcns  and  answers  about  the  nature  and 

teaching  of  thinking  skills.  In  J.  B.  Baron  &  R.  J.  Sternberg  (Eds.) , 
Teaching  thinking:  Theory:  and  practice  (pp.  251-259) .  Vesa  Yoidc: 
Freeman. 

Sternberg,  R.  J.  (1987) .  Review  of  B.  Presseisen,  Thinking  skills. 
EducaUonal  Horizons.  151. 

Sternberg,  R.  J.  (1987).  Second  game:  A  schod's-eye  view  of 
Intelligenoe.  In  J.  A.  langer  (Ed.),  T;^nmipep,  literacy,  and 
culture:  Issues  of  society  and  schooling  (pp.  23-48) .  Norwood,  NT: 
Ablex. 


Sternberg,  R.  J.  (1987) .  Stalking  a  ncrying  target.  Review  of  R. 
Nickerson,  D.  Perkins,  &  E.  Smith,  The  teaching  of  thinking. 
irarv  Psychology.  32#  322-323. 


Sternberg,  R.  J.  (1987) .  Synopsis  of  a  triarchlc  theory  of  human 

intelligenoe.  In  S.  H.  Irvine  &  S.  E.  Newstead  (Eds.),  Intelligence 
and  cognition:  COntemporarv  frames  of  nefetienoe  (pp.  141-176) . 
Dordrecht,  The  Netherlands:  Martinus  Nijhoff. 


Sternberg,  R.  J.  (1987) .  Teaching  critical  thinking:  Eight  easy  ways  to 
fedl  before  you  begin.  Rii  Delta  Ratoan.  456-459. 

Sternberg,  R.  J.  (1987) .  Teaching  intelligenoe:  The  application  of 

cognitive  psychology  to  the  iaprovenent  of  intellectual  skills.  In  J. 
B.  Baron  &  R.  J.  Sternberg  (Eds.),  Teaching  thinking  ddlls:  Theory 
and  practice  (pp.  182-218) .  New  York:  ireeoian. 

Sternberg,  R.  J.  (1987) .  Teaching  intelllgenoe:  A  trlarchic  model, 
m  D.  N.  Perkins,  J.  Lochhead,  &  J.  C.  Bishop  (Eds.),  Thinking:  The 
second  international  oonf erenoe  (pp.  53-60).  Hillsdale,  NT:  Erlbaum. 


Sternberg,  R.  J.  (1987) .  Three  portraits  of  intellectual  giftedness. 
Glftfd  Students  Institute  Quarterly.  i2,  A1-A5. 


CNR  Ocntxact  N0001485K0589  Final  Report,  Page  112 


Sterziberg,  R.  J.  (1987) .  Hie  trlarchlc  theory  of  human  intelligenoe; 

A  frameMork  for  the  understanding,  investigation,  testing,  and 
training  of  intelligenoe.  In  J.  Richardson,  M.  W.  lysenck,  &  D.  W. 
Piper  (Eds.),  student  leaTT^^nq«  Ref^earch  in  education  and  oocmitive 
psychology  (pp.  357-374) .  london:  SRHE  &  Hie  Ogen  university  Press. 

Sternberg,  R.  J.  (1987) .  A  unified  theoretical  perspective  on  autism. 

In  D.  Oohen  &  A.  M.  Oonnellan  (Eds.),  Wan^iw-tV  of  autism  and  pervasive 
develoanented  disorders  (pp.  690-696) .  New  Yoric:  Wiley. 

Sternberg,  R.  J.  (1987) .  A  unified  theory  of  intellectual 

exceptionadity.  In  J.  D.  Day  &  J.  G.  Barkowski  (Eds.),  Intelligence 
and  exceptionalltvi  wew  Mr^Aans  for  theory,  assessment,  and 
instructional  practices  (pp.  135-172).  Norwood,  NT:  Ablex. 

Sternberg,  R.  J.  (1987) .  Hie  uses  and  misuses  of  intelligence  testing. 
Educa^on  Wedc.  S^it.  23,  28,  22. 

Sternberg,  R.  J. ,  &  Berg,  C.  A.  (1987) .  What  are  theories  of  adult 
intellectual  development  theories  of?  In  C.  Schooler  &  K.  Wsumer 
Schaie  (Eds.),  Oognitlve  functioning  and  social  structure  over  the, 
life  course  (pp.  3-23) .  Norwood,  N7:  Ablex. 

Sternberg,  R.  J. ,  &  Davidson,  J.  E.  (1987) .  Teaching  thinking  to  college 
students:  Seme  lessons  learned  from  eaqperlence.  Teaching  Hiinkina 
EPOblgBL  S<2lvtag»  May/June,  1-2,  10-U. 

Sternberg,  R.  J.,  &  Dobson,  D.  M.  (1987).  Resolving  inbetpersonad 
conflicts:  An  analysis  of  stylistic  consistency.  Journal  of 
Personality  an^  soeigi  Psychology.  SZ,  794-812. 

f^tgner,  R.  K. ,  &  Sternberg,  R.  J.  (1987) .  Executive  control  in  reading 
oonpop^iension.  In  B.  K.  Britton  &  S.  M.  Glynn  (Eds. ) ,  Executive 
control  processes  in  reading  (pp.  1-21).  Hillsdale,  NT:  Erlbaum. 


I§88 


Qkagakl,  L. ,  &  Sternberg,  R.  J.  (1988) .  Uhwre^ing  giftedness.  In  G. 
Kanselaar,  J.  L.  van  der  Linden,  &  A.  Pennings  (Eds. ) ,  Tnd-ividuai 
differences  in  giftedness;  Identification  and  education  (pp.  30-45). 
Louvadn,  Belgium:  Aoco. 


Sincoff ,  J.  B. ,  &  Sternberg,  R.  J.  (1988) .  Hie  develapmmit  of  verbal 
fluency  abilities  and  strategies  in  elementary-school-aged  children. 
Develcanental  Psychology.  24,  646-653. 


Sternberg,  R.  J.  (Ed.)  (1988).  Advances  in  the  psychology  of  human 
intelligence  (Vol.  4) .  Hillsdale,  NJ:  Erlbaum. 


Sternberg,  R.  J.  (1988) .  Beyond  IQ  testing,  watiopai  mrYpniir  spring. 

8-U. 


Sternberg,  R.  J.  (1988) .  Hie  development  of  intellectual  styles. 
Teaching  Hiinking  and  Problem  Solving.  IQ(2),  1-4. 


CMR  Ocsntract  N0001485K0589  Final  Peport,  Page  113 


Sternberg,  R.  J.  (1988) .  geheces:  A  rationale  fcsr  the  construct 

vali^tion  of  theories  and  tests  of  intelligenoe.  In  H.  Wainer  &  H. 

I.  Braun  (Eds.)  /  Test  validity  (pp.  61-75).  Hillsdale,  NT:  Erlhaum. 

Sternberg,  R.  J.  (1988) .  The  intellect:  Three  partraits  unveiled. 

Gifted  Oi-lldinen  Monthly.  9,  1-3. 

Sternberg,  R.  J.  (1988) .  Intellectual  development:  Psychcmetrlc  and 
Infamation-pirooesslng  2$proaches.  In  M.  Bomstein  &  M.  Lainb  (Eds. ) , 
Develccroental  psychology:  An  advanced  textbook  (2nd  ed. ,  pp. 

261-295) .  Hillsdade,  KT:  Erlbaum. 

Sternberg,  R.  J.  (1988) .  Intelligence.  In  R.  J.  Sternberg  &  E.  E.  Smith 
(Ec3s.),  The  TDSvcholociv  of  human  ^hrtioh'h  (pp.  267-308).  New  YorJc: 
canbridge  University  Press. 

Sternberg,  R.  J.  (1988) .  Intelligence  is  more  than  a  matter  of  "Hew 
nuch?"  Hunan  Intelligence  Int^niiatlonai  w<^ietter.  S/4,  2-4. 

Sternberg,  R.  J.  (1988) .  Lessons  fren  the  life  ^>an:  Vhat  theorists  of 
intellectual  development  among  children  can  learn  from  their 
counterparts  stuctying  adults.  In  E.  M.  Hetherlngton,  R.  M.  Lamer,  & 
M.  Perlmutter  (Eds. ) ,  Child  develoonent  in  life-span  perspective  (pp. 
259-275).  Hillsdale,  NT:  Erlhaum. 

Sternberg,  R.  J.  (1988) .  The  male/fanale  difference  is  there:  Should  we 
care?  Behavioral  and  Brain  Sciences.  11(2),  210-211. 

Sternberg,  R.  J.  (Ed.)  (1988).  The  nature  of  creativity.  New  York: 
Cambridge  University  Press. 

Sternberg,  R.  J.  (1988) .  A  practiced  e^proach  to  practical  intelligenoe. 
Oontencorarv  Psychology.  13,  504-505. 

Sternberg,  R.  J.  (1988).  The  psychologist *s  ccpcanlon  (2nded.). 

New  York:  Cantoidge  university  Press. 

Sternberg,  R.  J.  (1988) .  Questioning  cognitive  psychology.  OonteBPorarv 
Psychology.  33,  206-207. 

Sternberg,  R.  J.  (1988) .  Review  of  The  long-term  retention  of  knowledge 
and  skills:  A  cognitive  and  instructicnaT  perspective.  Human 
Intelligence  intemat-ionai  Newsletter.  8/4,  10. 

Sternberg,  R.  J.  (1988) .  Review  of  H.  ^itz,  The  raising  of 

intelligence:  A  selected  historv  of  attempts  to  raise  retarded 
|n»eiHaenne.  American  Journal  of  Psychology.  IQl,  142-145. 

Sternberg,  R.  J.  (1988) .  Survival  of  the  fittest  in  theories  of 

creativity.  Journal  of  Social  and  Biological  Structures,  il,  154-155. 

Sternberg,  R.  J.  (1988) .  A  three-facet  model  of  creativity.  In  R.  J. 
Sternberg  (Ed. ) ,  The  nature  of  creativity  (pp.  125-147) .  New  York: 
canbridge  university  Press. 

Sternberg,  R.  J.  (1988).  The  triarchic  mind;  A  new  theory  of  human 
intelligence.  New  York:  Viking. 


CNR  Ocntract  N0001485K0589  Final  depart,  Page  114 


Sternberg,  R.  J. ,  &  Martin,  M.  (1988).  V3ien  teaching  thinking  does  not 
work,  vhat  goes  wrong?  Tteaenara  college  Record.  £§,  555-578. 

Sternberg,  R.  J. ,  &  Smith,  E.  E.  (Eds.)  (1988).  The  psvdioloav  of  human 
thcMcht.  New  York:  Cambridge  university  Press. 

Tardif ,  T. ,  &  Sternberg,  R.  J.  (1988) .  Oonclusicns:  Nhat  do  we  know 
about  creativity?  In  R.  J.  Sternberg  (Ed. ) ,  The  nature  of  creativity 
(pp.  429-440) .  New  York:  Cambridge  university  Press. 


In  press 

Ackeman,  P. ,  Sternberg,  R.  J. ,  &  Glaser,  R.  (Eds.)  (in  press).  Learning 
and  individual  d^fferenoes.  New  York:  Freeman. 


Kalmar,  D.  A. ,  &  Sternberg,  R.  J.  (in  press) .  Iheory  knitting: 
integrative  approach  to  theory  development.  ■ 


An 


logy. 


Kolligian,  J. ,  Jr,,  &  Sternberg,  R.  J.  (Eds.)  (in  press).  Peroeptions  of 
ocTOPetenoe  and  inootpetenoe.  New  Haven,  CT:  Yede  university  Press. 

Frensch,  P.  A. ,  &  Sternberg,  R.  J.  (in  press) .  Esgiertise  and  intelligent 
thinking:  Mhen  is  it  worse  to  know  better?  In  R.  J.  Sternberg  (Ed.) , 
Advecices  in  the  pevchnlngy  nf  I^Mman  intellioefnnft  (Vol.  5).  Hillsdale, 
NT:  Erlbaum. 


Slnooff ,  J. ,  &  Sternberg,  R.  J.  (in  press) .  Hie  development  of  cognitive 
skills.  In  A.  M.  Colley  &  J.  R.  Beech  (Eds. ) ,  The  aoa»ii«iti?n 
oerfomemoe  of  cognitive  skills.  Chichester,  England:  Wiley. 

Sternberg,  R.  J.  (Ed.)  (in  press).  Advances  in  the  psvcholoav  of  human 
intelligence  (Vol.  5).  Hillsdale,  NT:  Erlbaum. 


Sternberg,  R.  J.  (in  press) .  ^plying  cognitive  theory  to  the  testing 
and  teaching  of  intelligenoe.  Applied  Cognitive  Psychology. 

Sternberg,  R.  J.  (in  press) .  Behind  closed  doors:  unlocking  the 

oyst^es  of  human  intelligence.  In  J.  Brocianan  (Ed. ) ,  The  reality 
club  H.  New  York:  lYNX. 
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Section  3 
Presentations 


October,  1985 

An  infomal  presentation  at  the  Pstychological  Oorporation 
A  ooUoguiun  at  the  university  of  Delaware 
An  invited  address  to  the  Association  for  Gifted  and 

Talented  Education  at  the  Oonoord  Hotel  in  New  York:  State 
A  ooUoguiiM  at  Harvard  University 
A  ooUo^iium  at  Trill  gh  university 
The  keynote  address  to  the  New  York  State  Oonf erenoe 
on  Thinking  in  Albany 
A  workshop  in  Vernon,  Oomecticut 
A  talk  at  a  oonferenoe  at  Stanford  university 
Ihe  keynote  address  at  the  Michigan  State  Oonfereice 
on  Thinking  in  Detroit. 

A  colloquium  at  the  university  of  Michigan 

Novenber,  1985 

A  talk  to  a  growp  of  administrators  at  Southern  Connecticut 
State  university 

The  keynote  a>ddresa  at  the  National  Association  for 
Gifted  Children 

An  address  at  a  weekrlong  oonferenoe  on  cognition  in  Madrid, 

Spedn 

A  talk  at  the  meeting  of  the  Fsychoncnic  Society  in  Boston 
The  keynote  address  at  the  meeting  of  the  Pennsylvania 
Association  for  Stpervislon  and  Curriculum  Develo{iDent 

Deoenber,  1985 

An  invited  t2Qk  to  the  Kashington  State  Ccnf erenoe 
on  Thinking  in  Seattle 

A  ooUoguium  at  the  Center  for  Creative  leadership  in  Greensboro 
An  invited  talk  to  teaohers  in  Glastonbury,  Connecticut 
An  invited  talk  to  teachers  in  Southington,  Connecticut 
TVro  ooUoquia  at  Vanderbilt  university 

January,  1986 

A  wQikshop  on  thinking  skills  to  the  Nest  Hartford  School  System 
A  ooUoguium  on  intelligence  to  the  Bank  Street  College  of 
Education 

A  Keynote  etddress  on  teaching  thinking  to  the  Penn^lvania 
Association  for  Supervision  and  Curriculxan  Developooent 

February,  1986 

A  colloquium  on  intelligenoe  at  the  university  of  Texas  at  Austin 
A  colloquium  on  Intelligence  at  Rice  university  in  Houston 
A  talk  on  practical  int^llgenoe  for  business  vnriters  in  New  York 
A  tedk  on  practical  intelligenoe  at  Montclair  State  College  in 
Ipper  Montclair,  New  Jersey 

Talks  at  Lake  Washington  School  District,  Washington,  and  at  the 
annued  meeting  of  the  Association  for  Sipervision  and 
curriculum  DeveloExaent 
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March,  1986 

A  teQJc  at  Sacred  Heart  University,  Fairfield,  CT 
A  televrised  talk  to  teachers  in  the  State  of  Virginia 
A  workshop  in  Plainville,  CT 

An  invited  address  cn  intelligenoe  as  mental  self-government  at 
Gatlin^aerg  ocnferenoe 

April,  1986 

Keynote  addresses  and  workshops  at  Cheshire  (CT)  School  District 
and  Richmond  School  District  (BC,  Canada) 

Keynote  addresses  at  three  oonf erenoes  on  the  gifted  at  Michigan 
State  university,  Rutgers  (New  Jersey  State  Ocnferenoe  on  the 
Gifted) ,  and  BOCES  of  NSw  York 
A  ooUoguiian  at  Columbia  university  Teachers  College 
A  ooUoqLiium  at  Western  Connecticut  State  university 

May,  1986 

A  colloquium  at  Loyola  university  (Chicago) 

A  talk  at  the  New  School  for  Social  Research 
TaUcs  at  thinking  skills  oonferenoes  qxnsared  by  the  City  of  New 
York,  South  Dakota  State  University,  and  the  International 
Foundation  for  Learning  (Vanoouver,  BC) 

June,  1986 

Talks  on  testing  at  oonf ecenoes  Gponsored  by  ET5  and  by  the 
College  Board 

Several  talks  at  the  AERA  annual  meeting 

July,  1986 

A  talk  on  predlctiav'postdiction  at  the  CNR  Contractors  meeting 
in  Champaign,  IL 

Lectures  and  woikshaps  on  thinking  at  the  CarMmff  Institute 
(Amherst) ,  the  College  of  St.  Ihcsas  (St.  Paul) ,  the  Rotary 
avib  (Hainden,  CT) ,  Confratuts  (Storrs,  CT) ,  Fai^leld 
university,  an  ASCO  institute  in  Williamsburg,  Northwestern 
university,  and  Woodbury  (CT) . 

August,  1986 

A  colloquium  on  theory  of  intellectual  styles  at  Duke  university 
An  invited  address  and  2  panel  discussions  at  APA  in  Washington 
A  lecture  on  intelllgenoe  at  the  university  of  Leuven  (Belgium) 

An  invited  emidress  at  the  annual  meeting  of  the  German 
Psychological  Society  in  Heidelberg 

October,  1986 

A  workshop  on  critical  thinking  in  Newtown  (CT)  Hi^  School 
A  workshop  on  testing  intelligenoe  to  the  Connecticut  School 
Psychologists  Association 

A  talk  on  testing  for  college  asmisslons  at  the  College  Board 
annual  conference 

A  lecture  on  ay  theory  of  intelligenoe  in  Wilkasboro,  NC,  the 
site  for  the  pilot  Multidimensional  Ability  Test 
A  lecture  on  thinking  in  the  West  Irondequoit  School  District 
(rpper  NSfr*  York  State) . 
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November,  1986 

A  keynote  address  to  the  NSw  York  State  Beading  Association 

A  lecture  on  thinking  skills  to  textbook  writers  and  editors  at 
Baroourt  Brace  Jovanovich  International  headguartos  in 
Orlando 

A  talk  on  novelty  and  intelligenoe  at  the  Peychononic  Society 
meetings 

A  major  address  and  panel  discussion  at  a  CarsKlian  Conference  on 
thinking  skills  in  Toronto 

December,  1986 

An  all-day  workshop  on  thinking  at  the  university  of  California, 
Irving 


January,  1987 

An  invited  address  on  thinking  skills  at  the  annual  North 
Carolina  ASCD  meeting 

Talics  to  a  series  of  educational  consortia  in  New  York  State 
throuc^  N.Y.  Central  B0CE5. 

February,  1987 

CoUoguia  at  Quinnipiac  College  (New  Haven) ,  the  Education  School 
of  the  university  of  Pennsylvania,  and  the  University  of 
Washington  Ptychology  Department 

A  talk  to  school  district  officials  in  Seattle  throu^  a  program 
arranged  by  Antioch  university 

A  tadk  at  an  ETS  meeting  on  graduate  student  success 

March,  1987 

CoUoguia  at  the  university  of  Saskatchewan  and  of  Begina 

WOricshope  at  the  university  of  Rierto  Bioo  in  Bio  Piedras  and  in 
Cayjey 

A  trOk  at  a  conference  on  schooling  and  intelligenoe  at  the 
university  of  Michigan 

A  t2dk  at  a  meeting  on  personnel  and  instruction  in  San  Antonio 

Waokshops  on  thinking  skills  in  Tnmbull  (CT) ,  Washtinaw  County 
(MI) ,  and  Mamaroneck,  NY 

Ihe  keyrnte  address  and  workshop  at  a  conference  on  giftedness  at 
Arlington,  TX 

A  talk  at  the  ONR  Contractors'  meeting 

April,  1987 

CoUoguia  on  sy  research  on  inteUigenoe  at  the  Uhlversity  of 
Texas,  San  Antonio,  the  Uhlversity  of  Maryland,  and  the 
university  of  Pittebur^ 


) 
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May,  1987 

A  ooUoqcdum  at  the  dty  Uhiverslty  of  New  York 

Talks  at  a  N^  Jersey  State  Oonference  on  Thinking  Skills,  the 
New  Yack  ASCD  annual  oonference,  and  the  Uhiverslty  of 
Minnesota  Round  Table 

June,  1987 

A  talk  at  the  annual  CNR  Ocntractors'  aieetlng 

A  workshop  in  the  Fedrfleld  (d)  schools 

J\dy,  1987 

An  invited  address  on  ay  views  of  intelllgenoe  and  its 

developoent  to  the  International  Society  for  the  Study  of 
Behavior  Developoent  in  Japan 

TWO  lectures  on  the  nature  of  intelligenoe  at  the  university  of 
Hawed! 

August,  1987 

An  invited  eoddress  on  the  nature  of  giftedness  at  the  world 
Gifted  Oonference  in  Salt  Lake  City 

A  three  day  workshcp  on  thinking  Skills  sponsored  by  AEEA  in  San 
Francisco 

An  invited  address  on  Intelligenoe  testing  and  a  synposium  on 
aging  at  APA  annual  conference  in  New  Yoodc 

September,  1987 

A  colloquium  at  the  State  university  of  New  York  in  Buffalo 

October,  1987 

A  keynote  eddress  at  the  Florida  State  Gifted  Association  annual 
meeting 

A  series  of  lectures  to  educators  in  the  St.  Louis  Special  School 
District 

Wbrkshops  in  the  New  Fairfield  and  South  Windsor  (CT)  School 
Districts 

November,  1987 

A  talk  at  the  annual  meeting  of  the  Psychoncmic  Society 

A  colloquium  at  Camegie-Mallon  university 

The  keynote  address  at  the  Tennessee  and  Illinois  State  Gifted 
Associations  annual  meetings 

A  workshop  to  hi^-level  educators  at  Research  for  Better  Schools 
in  Ihiladelphia 

Deoeafcer,  1987 

A  colloquium  at  the  university  of  Cincinnati 


February,  1988 

A  ooUoqulm  at  the  university  of  Toledo 
A  colloquium  at  Florida  State  university 
An  invited  address  on  testing  to  the  Naticnal  Association  of 
Independent  Schools  in  NYC 
A  universlty>wlde  lecture  at  Bowdoin  College 
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March,  1988 

An  address  to  a  consortium  of  educators  frcm  the  Chicago  region 
A  talk  to  teachers  at  the  Bron»dlle  School 
An  2K3dress  to  scholars  at  the  university  of  Texas,  Austin,  in  a 
conference  on  new  directions  in  testing  organized  by  Earl  Hunt 
and  Don  Foss 

April,  1988 

A  ooUogulvim  at  the  university  of  Rochester 
A  ooUoqulum  at  the  university  of  Wisconsin,  River  Falls 
A  t2Qk  at  a  oonferenoe  at  Calgary 
A  talk  at  Colvmhla  High  School  in  Maplewood,  MJ 

May,  1988 

A  talk  at  a  high  school  in  Illinois 

June,  1988 

A  talk  to  testing  perscnnsl  of  the  U.S.  Enplcynent  Sezvloe 

July,  1988 

CoUoguia  and  talks  at  an  International  Conference  on  Individual 
Differences  in  Cognition  in  Ramat  Gan,  Isreal  and  at 
university  of  Haifa 

Talks  at  the  Hhtlonal  institute  of  Testing  and  Eveduatlon  in 
Jenosalen  and  the  welzman  Institute  of  Science 
A  we^-long  institute  on  thinking  skills  at  Yale 
A  talk  to  Oanfratute  (an  institute  for  teachers  of  the  gifted) 

August,  1988 

IVk)  talks  at  the  American  Psychologlced  Association  meetings  in 
Atlanta 

A  talk  at  the  International  Conference  on  Intelligence  censored 
by  the  Australian  government 

A  talk  at  the  International  Conference  on  Thlilklng  Skills  in 
Edmonton,  Alherta 


